Marakana Android Internals

Aleksandar (Saša) Gargenta, Ken Jones, Marko Gargenta

August 30, 2012

2012 Markana, Inc. All rights reserved.

Overview

Android Stack

images/AndroidStack.svg

The purpose of this module is to explain the anatomy of the Android OS. We will walk through the Android stack, starting from the bottom and moving up:

   Android Linux Kernel Layer

images/AndroidStack-Linux.svg
Android Linux Kernel Layer

Overview

The following are some of the changes/additions Android makes to the Linux kernel.

Binder IPC

Ashmem (Anonymous SHared MEMory)

Pmem

Wakelocks

Early Suspend

Alarm

Low Memory Killer (a.k.a. Viking Killer)

Logger

Paranoid Network Security

Other Kernel Changes

   Android User-Space Native Layer

images/AndroidStack-NativeLayer.svg
Android User-Space Native Layer

Bionic Library

Changes From BSD libc

User-space Hardware Abstraction Layer (HAL)

Native Daemons

Flingers

Surface Flinger
  • SurfaceFlinger is Android’s system-wide screen composer that draws into standard Linux frame-buffer (/dev/fb0)
  • Apps draw (in 2D or 3D) into "windows", which are implemented as double-buffered Surface objects backed by the surface flinger
    • Front-buffer used for composition, back-buffer for drawing
    • Buffers are flipped after drawing
      • Minimal buffer copying
      • Avoids flickers and artifacts as the front-buffer is always available for composition
  • Surface flinger expects the video driver to offer:
    • A linear address space of mappable memory
      • Video memory is mmap()'ed to process address-space for direct writing
      • Enough video memory for twice the physical screen area
        • Otherwise, regular system memory has to be used for buffering, and is copied on flips (slow!)
    • Support for RGB 565 pixel format
Audio Flinger
  • AudioFlinger is Android’s system-wide audio stream routing engine/mixer and audio input capture facility
    • Sits on top of device-specific libaudio.so implementation, which usually simply bridges to ALSA
  • To play audio, apps send uncompressed mono/stereo PCM streams to audio flinger (usually via MediaPlayer)
    • Streams include ringtones, notifications, voice calls, touch tones, key tones, music
    • Audio flinger routes these streams to various outputs (earpiece, speakers, Bluetooth)
  • To capture audio, apps request access to uncompressed input path managed by the audio flinger (usually via MediaRecorder)

Function Libraries

Dalvik

images/Stack-Dalvik-Dan.png
images/Dalvik-LicensePlate.png

   Android Application Framework Layer

images/AndroidStack-ApplicationFramework.svg
Android Application Framework Layer

Overview

images/AndroidFooService.svg

Activity Manager Service

Package Manager Service

Power Manager Service

Alarm Manager Service

Notification Manager Service

Keyguard Manager Service

Location Manager Service

Sensor Manager Service

Search Manager Service

Vibrator Manager Service

Connectivity Manager Service

Wifi Manager Service

Telephony Manager Service

Input Method Manager Service

UI Mode Manager Service

Download Manager Service

Storage Manager Service

Audio Manager Service

Window Manager Service

Layout Inflater Manager Service

Resource Manager Service

Additional Manager Services

   Android Applications Layer

images/AndroidStack-Applications.svg
Android Applications Layer

Android Built-in Applications

Android Built-in Content Providers

Android Built-in Input Methods

Android Built-in Wallpapers

   Review Questions

  1. What are the four layers of the Android Stack?

  2. What is the purpose of the Linux Kernel? How does Android use it?

  3. What’s missing in Android for it to be considered a more traditional Linux distribution?

  4. How is the Linux kernel on Android different?

  5. Name at least five Android Linux kernel extensions.

  6. What is Binder?

    1. What does Binder do and why do we need it?

    2. How is Binder exposed to the user-space?

    3. How does Binder compare to other similar mechanisms?

  7. What is Ashmem?

    1. Why do we need Ashmem?

    2. How do applications use Ashmem?

    3. How does Ashmem compare to other similar mechanisms?

  8. How does the power management on Android compare to traditional Linux distributions?

    1. What are wake locks?

    2. Where in the stack are the wake locks used?

    3. How do applications get access to wake locks?

    4. What is the purpose of early suspend feature of the Linux kernel?

  9. What is the purpose of the alarm driver?

  10. What are the two opposing objectives on Android when it comes with memory management?

    1. What is Android’s first line of defense when it comes to memory management?

    2. What is the low-memory-killer and what does it do?

    3. What are the relative levels of priority of application processes on Android?

    4. How does the low-memory-killer know who to kill and when?

    5. How can applications avoid being low-memory-killed?

  11. What is the purpose of the logger on Android?

    1. Name at least three log destinations.

    2. Where is the log information stored?

  12. What is the purpose of the CONFIG_ANDROID_PARANOID_NETWORK kernel option on Android?

  13. Name at least three sub-layers of the native layer.

  14. What is Bionic, why do we need it, and how does it differ from its alternatives?

  15. What are the two main purposes of user-space HAL on Android?

    1. How is the user-space HAL on Android exposed to the layers above it?

    2. Name at least three classes of devices exposed by user-space HAL.

  16. Name at least five native daemons and explain what they do.

    1. What is the purpose of ueventd on Android?

  17. What are the two "flingers" on Android?

  18. Name at least three "function" libraries on Android.

  19. What is the name of the Android’s media framework?

  20. What is the purpose of Dalvik on Android?

    1. How does Dalvik differ from its alternatives?

    2. What is the purpose of zygote?

    3. What was added to Dalvik in Froyo (Android 2.2)?

  21. What’s inside the Android Application Framework layer?

  22. What is the purpose of system services on Android? What value do they add?

  23. What is the purpose of managers for system services?

  24. What is the purpose of servicemanager daemon?

  25. Name at least five system services on Android.

  26. What does ActivityManagerService do?

  27. What does PackageManagerService do?

  28. What does the PowerManagerService do?

  29. What does the AlarmManagerService do?

  30. What does the KeyguardManagerService do?

  31. What does the InputMethodManagerService do?

  32. What is Android CDD and why do we need it?

  33. What’s inside an APK?

  34. What’s the difference between system and non-system apps?

  35. What are the different classes of applications that ship with Android?

  36. Name at least three "special" applications on Android?

Android Native Development Kit (NDK)

   What is in NDK?

   Why NDK?

   Java Native Interface (JNI)

JNI Overview

In this module, we’ll explore the following topics:

Note

JNI can also be used to invoke Java code from within natively-written applications - such as those written in C/C++.

In fact, the java command-line utility is an example of one such application, that launches Java code in a Java Virtual Machine.

JNI Components

JNI By Example

  1. We start by creating a Java class with one or more native methods

    src/com/marakana/jniexamples/Hello.java:
    package com.marakana.jniexamples;

public class Hello {

    public static native void sayHi(String who, int times); // 1

    static {
        System.loadLibrary("hello"); // 2
    }

    public static void main(String[] args) {
        sayHi(args[0], Integer.parseInt(args[1])); // 3
    }
}
    1 The method sayHi(String, int) will be implemented in C/C+ in separate files, which will be compiled into a shared library.
    2 Load the shared library by its logical name. The actual name is system-dependent: libhello.so (on Linux/Unix), hello.dll (on Windows), and libhello.jnilib (Mac OSX).
    3 Here we call our native method as a regular Java method.

  2. Compile the Java code 

    $ mkdir -p bin
$ javac -d bin/ src/com/marakana/jniexamples/Hello.java

  3. Using the javah tool, we generate the C header file from the compiled com.marakana.jniexamples.Hello class: 

    $ mkdir -p jni
$ javah -jni -classpath bin -d jni com.marakana.jniexamples.Hello

  4. Observe the generated C header file:

    jni/com_marakana_jniexamples_Hello.h:
    /* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_marakana_jniexamples_Hello */

#ifndef _Included_com_marakana_jniexamples_Hello
#define _Included_com_marakana_jniexamples_Hello
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     com_marakana_jniexamples_Hello
 * Method:    sayHi
 * Signature: (Ljava/lang/String;I)V
 */
JNIEXPORT void JNICALL Java_com_marakana_jniexamples_Hello_sayHi
  (JNIEnv *, jclass, jstring, jint);

#ifdef __cplusplus
}
#endif
#endif
    Note Method names resolve to C functions based on a pre-defined naming strategy: the prefix Java_, followed by a mangled fully-qualified class name, followed by an underscore ("_") separator, followed by a mangled method name. For overloaded native methods, two underscores ("__") followed by the mangled argument signature.

  5. Provide the C implementation:

    jni/com_marakana_jniexamples_Hello.c:
    #include "com_marakana_jniexamples_Hello.h"

JNIEXPORT void JNICALL Java_com_marakana_jniexamples_Hello_sayHi
  (JNIEnv *env, jclass clazz, jstring who, jint times) {
  const char *name = (*env)->GetStringUTFChars(env, who, NULL);
  if (name != NULL) {
    jint i;
    for (i = 0; i < times; i++) {
      printf("Hello %s\n", name);
    }
    (*env)->ReleaseStringUTFChars(env, who, name);
  }
}
    Note Most of the time, we cannot just use Java data types directly in C. For example, we have to convert java.lang.String to char * before we can effectively use it in C.
    Note This code assumes: #define NULL ((void *) 0)

  6. Compile the shared library 

    $ mkdir -p libs
$ gcc -o libs/libhello.jnilib -lc -shared \
    -I/System/Library/Frameworks/JavaVM.framework/Headers \
    jni/com_marakana_jniexamples_Hello.c
$ file libs/libhello.jnilib
libs/libhello.jnilib: Mach-O 64-bit dynamically linked shared library x86_64
    Note On Unix/Linux, compile as:
    gcc -o libs/libhello.so -lc -shared -fPIC -I$JAVA_HOME/include jni/com_marakana_jniexamples_Hello.c

  7. Run our code 

    $ java -Djava.library.path=libs -classpath bin com.marakana.jniexamples.Hello Student 5
Hello Student
Hello Student
Hello Student
Hello Student
Hello Student
    Note Instead of specifying -Djava.library.path=libs, we could have preceded our java command with export LD_LIBRARY_PATH=libs.
    Tip Common mistakes resulting in java.lang.UnsatisfiedLinkError usually come from incorrect naming of the shared library (O/S-dependent), the library not being in the search path, or wrong library being loaded by Java code.

Native Method Arguments

Primitive Type Mapping

Reference Type Mapping

Global and Local References

Using Strings

Arrays

Reflection

Exceptions

   Using NDK

images/NDK.svg
NDK Process

   NDK and JNI by Example

Fibonacci - Java Native Function Prototypes

We start off by defining C function prototypes as native Java methods (wrapped in some class):

FibonacciNative/src/com/marakana/android/fibonaccinative/FibLib.java
package com.marakana.android.fibonaccinative;

import android.util.Log;

public class FibLib {
    private static final String TAG = "FibLib";

    private static long fib(long n) {
        return n <= 0 ? 0 : n == 1 ? 1 : fib(n - 1) + fib(n - 2);
    }

    // Recursive Java implementation of the Fibonacci algorithm
    // (included for comparison only)
    public static long fibJR(long n) {
        Log.d(TAG, "fibJR(" + n + ")");
        return fib(n);
    }

    // Function prototype for future native recursive implementation
    // of the Fibonacci algorithm
    public native static long fibNR(long n);


    // Iterative Java implementation of the Fibonacci algorithm
    // (included for comparison only)
    public static long fibJI(long n) {
        Log.d(TAG, "fibJI(" + n + ")");
        long previous = -1;
        long result = 1;
        for (long i = 0; i <= n; i++) {
            long sum = result + previous;
            previous = result;
            result = sum;
        }
        return result;
    }

    // Function prototype for future iterative recursive implementation
    // of the Fibonacci algorithm
    public native static long fibNI(long n);

    static {
        // as defined by LOCAL_MODULE in Android.mk
        System.loadLibrary("com_marakana_android_fibonaccinative_FibLib");
    }
}

Fibonacci - Function Prototypes in a C Header File

We then extract our C header file with our function prototypes:

  1. On the command line, change to your project’s root directory 

    $ cd /path/to/workspace/FibonacciNative

  2. Create jni sub-directory 

    $ mkdir jni

  3. Extract the C header file from com.marakana.android.fibonaccinative.FibLib class: 

    $ javah -jni -classpath bin/classes -d jni com.marakana.android.fibonaccinative.FibLib
    Note Prior to ADT r14, compiled class files were kept directly in the bin/ directory, so in our javah command we would’ve used -classpath bin instead.

  4. Check out the resulting file:

    FibonacciNative/jni/com_marakana_android_fibonaccinative_FibLib.h
    /* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_marakana_android_fibonaccinative_FibLib */

#ifndef _Included_com_marakana_android_fibonaccinative_FibLib
#define _Included_com_marakana_android_fibonaccinative_FibLib
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     com_marakana_android_fibonaccinative_FibLib
 * Method:    fibNR
 * Signature: (J)J
 */
JNIEXPORT jlong JNICALL Java_com_marakana_android_fibonaccinative_FibLib_fibNR
  (JNIEnv *, jclass, jlong);

/*
 * Class:     com_marakana_android_fibonaccinative_FibLib
 * Method:    fibNI
 * Signature: (J)J
 */
JNIEXPORT jlong JNICALL Java_com_marakana_android_fibonaccinative_FibLib_fibNI
  (JNIEnv *, jclass, jlong);

#ifdef __cplusplus
}
#endif
#endif
    Note The function prototype names are name-spaced to the classname they are found in.

Fibonacci - Provide C Implementation

We provide the C implementation of com_marakana_android_fibonacci_FibLib.h header file:

FibonacciNative/jni/com_marakana_android_fibonaccinative_FibLib.c
/* Include the header file that was created via "javah -jni" command */
#include "com_marakana_android_fibonaccinative_FibLib.h"
#include <android/log.h>

/* Recursive implementation of the fibonacci algorithm (in a helper function) */
static jlong fib(jlong n) {
    return n <= 0 ? 0 : n == 1 ? 1 : fib(n - 1) + fib(n - 2);
}

/* Actual implementation of JNI-defined `fibNR` (recursive) function */
JNIEXPORT jlong JNICALL Java_com_marakana_android_fibonaccinative_FibLib_fibNR
  (JNIEnv *env, jclass clazz, jlong n) {
    __android_log_print(ANDROID_LOG_DEBUG, "FibLib.c", "fibNR(%lld)", n);
    return fib(n);
}

/* Actual implementation of JNI-defined `fibNI` (iterative) function */
JNIEXPORT jlong JNICALL Java_com_marakana_android_fibonaccinative_FibLib_fibNI
  (JNIEnv *env, jclass clazz, jlong n) {
    jlong previous = -1;
    jlong result = 1;
    jlong i;
    __android_log_print(ANDROID_LOG_DEBUG, "FibLib.c", "fibNI(%lld)", n);
    for (i = 0; i <= n; i++) {
        jlong sum = result + previous;
        previous = result;
        result = sum;
    }
    return result;
}

Fibonacci - An Alternative Implementation (C++)

We could also use an alternative mechanism of linking native-code to managed code by pre-registering our functions. This leads to earlier detection of method-function mismatch issues, a slight performance improvement, and spares us the redundancy of the header file and the use of the javah command.

FibonacciNative/jni/com_marakana_android_fibonaccinative_FibLib.cpp
#include <jni.h>
#include <android/log.h>

namespace com_marakana_android_fibonaccinative {
    static jlong fib(jlong n) {
        return n <= 0 ? 0 : n == 1 ? 1 : fib(n - 1) + fib(n - 2);
    }

    static jlong fibNR(JNIEnv *env, jclass clazz, jlong n) {
        __android_log_print(ANDROID_LOG_DEBUG, "FibLib.c", "fibNR(%lld)", n);
        return fib(n);
    }

    static jlong fibNI(JNIEnv *env, jclass clazz, jlong n) {
        jlong previous = -1;
        jlong result = 1;
        jlong i;
        __android_log_print(ANDROID_LOG_DEBUG, "FibLib.c", "fibNI(%lld)", n);
        for (i = 0; i <= n; i++) {
            jlong sum = result + previous;
            previous = result;
            result = sum;
        }
        return result;
    }

    static JNINativeMethod method_table[] = {
            { "fibNR", "(J)J", (void *) fibNR },
            { "fibNI", "(J)J", (void *) fibNI }
    };
}

using namespace com_marakana_android_fibonaccinative;

extern "C" jint JNI_OnLoad(JavaVM* vm, void* reserved) {
    JNIEnv* env;
    if (vm->GetEnv(reinterpret_cast<void**>(&env), JNI_VERSION_1_6) != JNI_OK) {
        return JNI_ERR;
    } else {
        jclass clazz = env->FindClass("com/marakana/android/fibonaccinative/FibLib");
        if (clazz) {
            jint ret = env->RegisterNatives(clazz, method_table, sizeof(method_table) / sizeof(method_table[0]));
            env->DeleteLocalRef(clazz);
            return ret == 0 ? JNI_VERSION_1_6 : JNI_ERR;
        } else {
            return JNI_ERR;
        }
    }
}
Note Most of the Android’s JNI-based shared libraries are built using this, "alternative", approach where the functions are pre-registered.

Fibonacci - Makefile

We need a Android.mk makefile, which will be used by NDK to compile our JNI code into a shared library:

FibonacciNative/jni/Android.mk
# Defines the root to all other relative paths
# The macro function my-dir, provided by the build system,
# specifies the path of the current directory (i.e. the
# directory containing the Android.mk file itself)
LOCAL_PATH := $(call my-dir)

# Clear all LOCAL_XXX variables with the exception of
# LOCAL_PATH (this is needed because all variables are global)
include $(CLEAR_VARS)

# List all of our C files to be compiled (header file
# dependencies are automatically computed)
LOCAL_SRC_FILES := com_marakana_android_fibonaccinative_FibLib.c

# The name of our shared module (this name will be prepended
# by lib and postfixed by .so)
LOCAL_MODULE := com_marakana_android_fibonaccinative_FibLib

# We need to tell the linker about our use of the liblog.so
LOCAL_LDLIBS += -llog

# Collects all LOCAL_XXX variables since "include $(CLEAR_VARS)"
#  and determines what to build (in this case a shared library)
include $(BUILD_SHARED_LIBRARY)
Note It’s easiest to copy the Android.mk file from another (sample) project and adjust LOCAL_SRC_FILES and LOCAL_MODULE as necessary
Note See /path/to/ndk-installation-dir/docs/ANDROID-MK.html for the complete reference of Android make files (build system)

Fibonacci - Compile Our Shared Module

Finally, from the root of our project (i.e. FibonacciNative/), we run ndk-build to build our code into a shared library (FibonacciNative/libs/armeabi/libcom_marakana_android_fibonacci_FibLib.so):

$ ndk-build
Compile thumb  : com_marakana_android_fibonaccinative_FibLib <= com_marakana_android_fibonaccinative_FibLib.c
SharedLibrary  : libcom_marakana_android_fibonaccinative_FibLib.so
Install        : libcom_marakana_android_fibonaccinative_FibLib.so => libs/armeabi/libcom_marakana_android_fibonaccinative_FibLib.so

Running ndk-build clean will clean all generated binaries.

Note The command ndk-build comes from the NDK’s installation directory (e.g. /path/to/android-ndk-r5b), so it’s easiest if we add this directory to our PATH.
Note The current version of NDK (at least up to r5b) on Windows depends on Cygwin (a Unix-like environment and command-line interface for Microsoft Windows), or specifically "shell" (bash) and "make" (gmake) programs available through Cygwin. To run ndk-build on Windows, we first need to run bash and then execute ndk-build. It is important that both c:\path\to\cygwin\bin and c:\path\to\ndk be defined in our Path.
Controlling CPU Application Binary Interface (ABI)
  • By default, the NDK will generate machine code for the armeabi (i.e. ARMv5TE with support for Thumb-1) ABI
    • The target application library is packaged as lib/armeabi/lib<name>.so
    • Upon installation, it is copied to /data/data/<package>/lib/lib<name>.so
  • We could add support for ARMv7-A (including hardware FPU/VFPv3-D16, Thumb-2, VFPv3-D32/ThumbEE, and SIMD/NEON) via APP_ABI in a separate Application.mk file
    • APP_ABI := armeabi-v7a
      • ARMv7-a only
      • Packaged as lib/armeabi-v7a/lib<name>.so
    • APP_ABI := armeabi armeabi-v7a x86
      • Builds three versions of the library, for ARMv5TE, ARMv7-a, and x86 in a single "fat binary"
      • Packaged as
        • lib/armeabi/lib<name>.so
        • lib/armeabi-v7a/lib<name>.so
        • lib/x86/lib<name>.so
      • The Android system knows at runtime which ABI(s) it supports
        • primary ABI for the device corresponds to the machine code of the system image
        • secondary ABI (optional) corresponds to to another ABI that is also supported by the system image
        • Upon installation, Android first scans lib/<primary-abi>/lib<name>.so then lib/<secondary-abi>/lib<name>.so and installs the first that it finds to /data/data/<package>/lib/lib<name>.so
  • As an alternative to Application.mk file, we could also specify APP_ABI on the command-line: 
    ndk-build "APP_ABI=armeabi armeabi-v7a x86"
  • Support for x86 ABI (i.e. IA-32 instruction set) was added in NDK r6 (July 2011)

Fibonacci - Client

We can now build the "client" of our library (in this case a simple activity) to use our FibLib library.

Fibonacci - String Resources
FibonacciNative/res/values/strings.xml
<?xml version="1.0" encoding="utf-8"?>
<resources>

    <string name="hello">Get Your Fibonacci Numbers Here!</string>
    <string name="fibJR">fibJR</string>
    <string name="fibJI">fibJI</string>
    <string name="fibNR">fibNR</string>
    <string name="fibNI">fibNI</string>
    <string name="app_name">FibonacciNative</string>
    <string name="button">Get Fibonacci Result</string>

</resources>
Fibonacci - User Interface (Layout)
screens/FibonacciNativeMainLayout.png
Fibonacci Native Main Layout
FibonacciNative/res/layout/main.xml
<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:orientation="vertical" >

    <!-- This is just a simple title ("Get Your Fibonacci Here!")  -->
    <TextView
        android:layout_width="fill_parent"
        android:layout_height="wrap_content"
        android:gravity="center"
        android:text="@string/hello"
        android:textSize="25sp" />

    <!-- This is the entry box for our number "n"  -->
    <EditText
        android:id="@+id/input"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:ems="10"
        android:gravity="right"
        android:inputType="number" >
        <requestFocus />
    </EditText>

    <!-- This radio group allows the user to select the fibonacci implementation type  -->
    <RadioGroup
        android:id="@+id/type"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:orientation="horizontal" >

        <RadioButton
            android:id="@+id/type_fib_jr"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:text="@string/fibJR" />

        <RadioButton
            android:id="@+id/type_fib_ji"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:text="@string/fibJI" />

        <RadioButton
            android:id="@+id/type_fib_nr"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:text="@string/fibNR" />

        <RadioButton
            android:id="@+id/type_fib_ni"
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:layout_weight="1"
            android:text="@string/fibNI" />
    </RadioGroup>

    <!-- This button allows the user to trigger fibonacci calculation  -->
    <Button
        android:id="@+id/button"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:text="@string/button" />

    <!-- This is the output area for the fibonacci result  -->
    <TextView
        android:id="@+id/output"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:gravity="center"
        android:textSize="20sp" />

</LinearLayout>
Fibonacci - FibonacciActivity
FibonacciNative/src/com/marakana/android/fibonaccinative/FibonacciActivity.java
package com.marakana.android.fibonaccinative;

import android.app.Activity;
import android.app.ProgressDialog;
import android.os.AsyncTask;
import android.os.Bundle;
import android.os.SystemClock;
import android.text.TextUtils;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.Button;
import android.widget.EditText;
import android.widget.RadioGroup;
import android.widget.TextView;

public class FibonacciActivity extends Activity implements OnClickListener {

    private EditText input;

    private RadioGroup type;

    private TextView output;

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
        this.input = (EditText) super.findViewById(R.id.input);
        this.type = (RadioGroup) super.findViewById(R.id.type);
        this.output = (TextView) super.findViewById(R.id.output);
        Button button = (Button) super.findViewById(R.id.button);
        button.setOnClickListener(this);
    }

    public void onClick(View view) {
        String s = this.input.getText().toString();
        if (TextUtils.isEmpty(s)) {
            return;
        }

        final ProgressDialog dialog = ProgressDialog.show(this, "",
                "Calculating...", true);
        final long n = Long.parseLong(s);
        new AsyncTask<Void, Void, String>() {

            @Override
            protected String doInBackground(Void... params) {
                long result = 0;
                long t = SystemClock.uptimeMillis();
                switch (FibonacciActivity.this.type.getCheckedRadioButtonId()) {
                case R.id.type_fib_jr:
                    result = FibLib.fibJR(n);
                    break;
                case R.id.type_fib_ji:
                    result = FibLib.fibJI(n);
                    break;
                case R.id.type_fib_nr:
                    result = FibLib.fibNR(n);
                    break;
                case R.id.type_fib_ni:
                    result = FibLib.fibNI(n);
                    break;
                }
                t = SystemClock.uptimeMillis() - t;
                return String.format("fib(%d)=%d in %d ms", n, result, t);
            }

            @Override
            protected void onPostExecute(String result) {
                dialog.dismiss();
                FibonacciActivity.this.output.setText(result);
            }
        }.execute();
    }
}

Fibonacci - Result

screens/FibonacciNativeResult.png
Fibonacci Native Result

   NDK’s Stable APIs

The header files for NDK stable APIs are available at /path/to/ndk/platforms/<android-platform>/<arch-name>/usr/include.

Android-specific Log Support

ZLib Compression Library

The OpenGL ES 1.x Library

The OpenGL ES 2.0 Library

The jnigraphics Library

The OpenSL ES native audio Library

The Android native application APIs

Caution With the exception of the libraries listed above, the native system libraries in the Android platform are not considered "stable" and may change in future platform versions. Unless our library is being built for a specific Android ROM, we should only make use of the stable libraries provided by the NDK.
Note All the header files are available under: /path/to/ndk-installation-dir/platforms/android-9/arch-arm/usr/include/
Note See /path/to/ndk-installation-dir/docs/STABLE-APIS.html for the complete reference of NDK’s stable APIs.

   Lab: NDK

Create a simple Android application that allows the end-user to log to Android’s logcat via functionality provided by /system/lib/liblog.so (i.e. you cannot use android.util.Log).

For example, this could be your LogLib:

package com.marakana.android.loglib;
public class LogLib {
    public static native void log(int priority, String tag, String msg);
    /* You'll need to load your library here */
}

The basic scaffolding for your application, along with the UI is already provided:

You just need to import it into Eclipse (as an existing project into your workspace) and then add the JNI components.

To test, look for your message to appear in adb logcat.

Warning Make sure that you enter something for both the tag and the message when testing your application.

As a bonus:

Tip Don’t forget to convert tag and msg strings from the Java format (jstring) to native format (char *) before trying to use them in int __android_log_write(int prio, const char *tag, const char *text). Also, don’t forget to free the native strings before returning from the native method. Finally, don’t forget to tell the linker about your use of the log library.

The solution is provided:

   Review Questions

  1. Who uses JNI on Android and why?

  2. What is the NDK?

  3. What is JNI?

  4. What is the purpose of jni.h?

  5. What is the purpose of javah tool?

  6. What are the first two arguments to all native functions that model Java methods declared as native?

  7. What is jlong and why do we need it (and its cousins)?

  8. How are boolean-s modeled in native code?

  9. How are java.lang.Object-s modeled in native code?

  10. What’s the definition of NULL?

  11. What is the difference between local and global references?

  12. What is the difference between Java strings and C "strings"?

  13. What is the difference between GetStringChars and GetStringUTFChars?

  14. How do modified UTF-8 strings differ from normal UTF-8 strings?

  15. Why do we have to call ReleaseStringChars or ReleaseStringUTFChars when we are done with the C strings?

  16. Name at least five array operations?

  17. What is the purpose of mode attribute in ReleaseByteArrayElements(…, mode) (and its cousins)?

  18. What is the signature of void f(int[] a, String s, long n)?

  19. What happens when an exception is thrown by native code?

  20. What is the significance of extern "C" jint JNI_OnLoad(JavaVM* vm, void* reserved) function?

  21. What can we do with jint RegisterNatives(JNIEnv*, jclass, const JNINativeMethod*, jint) function?

  22. What does LOCAL_MODULE specify in Android.mk?

  23. How do we control the ABI when compiling NDK code?

  24. Name at least three stable NDK APIs.

Android Binder Inter Process Communication (IPC) with AIDL

   Why IPC?

   What is Binder?

images/BinderIPC.png

   What is AIDL?

   Limitations of Binder

   Building a Binder-based Service and Client by Example

images/FibonacciClientServiceArchitecture.svg

FibonacciCommon - Define AIDL Interface and Custom Types

FibonacciService - Implement AIDL Interface and Expose It To Our Clients

Expose our AIDL-defined Service Implementation to Clients

FibonacciClient - Using AIDL-defined Binder-based Services

   Async-IPC via Binder by Example

FibonacciCommon - Defining a oneway AIDL Service

FibonacciService - Implementing our async AIDL service

FibonacciClient - Implementing our async AIDL client

   Lab: Binder-based Service with AIDL

Create an AIDL-described ILogService that provides the following functionality:

package com.marakana.android.logservice;
public interface ILogService {
    public void log(LogMessage logMessage);
}

where LogMessage is defined as follows:

package com.marakana.android.logservice;
public class LogMessage … {
    …
    public LogMessage(int priority, String tag, String msg) {
        …
    }
    …
}

Create a simple Android client that allows the user to submit a LogMessage request to the remote ILogService running in a separate process.

You can borrow code (UI) from the basic LogNative application, which you can get:

Tip Your implementation could simply use android.util.Log.println(int priority, String tag, String msg) to do the logging.

The solution is provided:

   Review Questions

  1. Why do we need IPC on Android?

  2. What is Binder’s unit of data called?

  3. What is AIDL and when do we use it?

  4. Is the use of AIDL required when consuming or exposing bound services?

  5. Name at least five supported method parameter/return data types that we use in AIDL.

  6. Name at least three types that are not supported.

  7. What is android.os.Bundle?

  8. What is android.os.Parcelable and how do we implement it?

  9. What do we have to do with android.os.Parcelable classes before we can use them in AIDL interfaces?

  10. What is android.os.IBinder?

  11. What is special about file descriptors?

  12. What is the purpose of the directional flag?

  13. What is the purpose of the aidl tool?

  14. What is the purpose of the Stub and the Proxy?

  15. What is the Android library project and when do we use them?

  16. What is the purpose of android.app.Service in the context of bound services?

  17. How do we expose a bound service to other applications?

  18. What is the thread context of an binder service request?

  19. How and when do we bind to non-system services?

  20. What is the life-cycle dependency between bound services and their clients?

  21. What is the purpose of the oneway keyword and when do we use it?

  22. What do we have to worry about when handling call-backs from remote services?

Android Security

   Android Security Architecture

images/AndroidProcessSnapshot-NoPermissions.svg

   Application Signing

Platform Keys

   User IDs

   File Access

   Using Permissions

Top Ten Bad Permissions for Games

Using the following permissions will significantly lower the likelihood for an Android app/game to be featured in Google Play (from Google I/O 2012):

  1. android.permission.SEND_SMS and android.permission.RECEIVE_SMS

  2. android.permission.SYSTEM_ALERT_WINDOW

  3. com.android.browser.permission.READ_HISTORY_BOOKMARKS and com.android.browser.permission.WRITE_HISTORY_BOOKMARKS

  4. android.permission.READ_CONTACTS, android.permission.WRITE_CONTACTS, android.permission.READ_CALENDAR, android.permission.WRITE_CALENDAR

  5. android.permission.CALL_PHONE

  6. android.permission.READ_LOGS

  7. android.permission.ACCESS_FINE_LOCATION

  8. android.permission.GET_TASKS

  9. android.permission.RECEIVE_BOOT_COMPLETED

  10. android.permission.CHANGE_WIFI_STATE

Using existing application components that have the necessary permissions

   Permission Enforcement

There are a number of trigger points for security/permission checks:

Kernel / File-system Permission Enforcement

UID-based Permission Enforcement

Static Permission Enforcement

Enforcing Permissions Dynamically

images/AndroidSecurityCallFlow.svg

   Declaring Custom Permissions

Before we can enforce our own permissions, we have to declare them using one or more <permission> in

AndroidManifest.xml
<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.marakana.android.myapp" >
  <permission
    android:name="com.example.app.DO_X"
    android:label="@string/do_x_label"
    android:description="@string/do_x_desc"
    android:permissionGroup="android.permission-group.PERSONAL_INFO"
    android:protectionLevel="dangerous" />
  …
</manifest>

Permission Components

Adding Custom Permissions Dynamically

   Custom Permissions by Example

Static Permission Enforcement

Here, we want to restrict access to the com.marakana.android.fibonacciservice.FibonacciService to applications (i.e. clients) that hold USE_FIBONACCI_SERVICE custom permission

  1. We start by by creating a custom permission group (making sure that we name-space it):

    FibonacciService/res/values/strings.xml:
    <?xml version="1.0" encoding="utf-8"?>
<resources>
    …
    <string name="fibonacci_permissions_group_label">Fibonacci Permissions</string>
    …
</resources>
    FibonacciService/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <permission-group
        android:name="com.marakana.android.fibonacciservice.FIBONACCI_PERMISSIONS"
        android:label="@string/fibonacci_permissions_group_label" />
    …
</manifest>
    Note This permission group is optional - as we could instead use one of the already provided groups

  2. Next, we create a custom permission (again, making sure that we name-space it), while taking advantage of our newly-created permission group:

    FibonacciService/res/values/strings.xml:
    <?xml version="1.0" encoding="utf-8"?>
<resources>
    …
    <string name="use_fibonacci_service_permission_label">use fibonacci service</string>
    <string name="use_fibonacci_service_permission_description">
      applications with this permissions get fibonacci results for free
    </string>
    …
</resources>
    FibonacciService/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <permission-group …/>
    <permission
        android:name="com.marakana.android.fibonacciservice.USE_FIBONACCI_SERVICE"
        android:description="@string/use_fibonacci_service_permission_description"
        android:label="@string/use_fibonacci_service_permission_label"
        android:permissionGroup="com.marakana.android.fibonacciservice.FIBONACCI_PERMISSIONS"
        android:protectionLevel="dangerous" />
    …
</manifest>

  3. Now we can statically require the permission on our FibonacciService service:

    FibonacciService/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <permission-group …/>
    <permission …/>
    <application …>
        <service
            android:name=".FibonacciService"
            android:permission="com.marakana.android.fibonacciservice.USE_FIBONACCI_SERVICE" >
            …
        </service>
    </application>
    …
</manifest>

  4. If we now re-run the FibonacciService and re-run the FibonacciClient, we will notice that the client will fail to launch and adb logcat will show something like: 

    …
W/ActivityManager(   85): Permission Denial: Accessing service ComponentInfo{com.marakana.android.fibonacciservice/com.marakana.android.fibonacciservice.FibonacciService} from pid=540, uid=10043 requires com.marakana.android.fibonacciservice.USE_FIBONACCI_SERVICE
D/AndroidRuntime(  540): Shutting down VM
W/dalvikvm(  540): threadid=1: thread exiting with uncaught exception (group=0x409c01f8)
E/AndroidRuntime(  540): FATAL EXCEPTION: main
E/AndroidRuntime(  540): java.lang.RuntimeException: Unable to resume activity {com.marakana.android.fibonacciclient/com.marakana.android.fibonacciclient.FibonacciActivity}: java.lang.SecurityException: Not allowed to bind to service Intent { act=com.marakana.android.fibonaccicommon.IFibonacciService }
E/AndroidRuntime(  540):    at android.app.ActivityThread.performResumeActivity(ActivityThread.java:2444)
…
E/AndroidRuntime(  540):    at dalvik.system.NativeStart.main(Native Method)
E/AndroidRuntime(  540): Caused by: java.lang.SecurityException: Not allowed to bind to service Intent { act=com.marakana.android.fibonaccicommon.IFibonacciService }
E/AndroidRuntime(  540):    at android.app.ContextImpl.bindService(ContextImpl.java:1135)
E/AndroidRuntime(  540):    at android.content.ContextWrapper.bindService(ContextWrapper.java:370)
E/AndroidRuntime(  540):    at com.marakana.android.fibonacciclient.FibonacciActivity.onResume(FibonacciActivity.java:65)
…
W/ActivityManager(   85):   Force finishing activity com.marakana.android.fibonacciclient/.FibonacciActivity
…

  5. Finally, we can give FibonacciClient a fighting chance by allowing it to use the USE_FIBONACCI_SERVICE permission:

    FibonacciClient/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <uses-permission android:name="com.marakana.android.fibonacciservice.USE_FIBONACCI_SERVICE"/>
    …
</manifest>

  6. We can now observe that our client is again able to use the service

  7. In the Emulator, if we go to Home → Menu → Manage apps → Fibonacci Client → PERMISSIONS, we should see the Fibonacci Permissions group and under it, use fibonacci service permission

Dynamic Permission Enforcement

Here, we want to restrict access to the com.marakana.android.fibonacciservice.IFibonacciServiceImpl's recursive operations (fibJR(long n) and fibNR(long n)) for n > 10 to applications (i.e. clients) that hold USE_SLOW_FIBONACCI_SERVICE custom permission

  1. Like before, we start off by creating a custom permission:

    FibonacciService/res/values/strings.xml:
    <?xml version="1.0" encoding="utf-8"?>
<resources>
    …
    <string name="use_slow_fibonacci_service_permission_label">
        use slow fibonacci service operations
   </string>
    <string name="use_slow_fibonacci_service_permission_description">
        applications with this permissions can melt the CPU and drain the battery
        by using slow fibonacci operations
    </string>
    …
</resources>
    FibonacciService/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <permission-group …/>
    <permission …/>
    <permission
        android:name="com.marakana.android.fibonacciservice.USE_SLOW_FIBONACCI_SERVICE"
        android:description="@string/use_slow_fibonacci_service_permission_description"
        android:label="@string/use_slow_fibonacci_service_permission_label"
        android:permissionGroup="com.marakana.android.fibonacciservice.FIBONACCI_PERMISSIONS"
        android:protectionLevel="dangerous" />
    …
</manifest>

  2. Next, we update our IFibonacciServiceImpl to enforce this permission dynamically - via a android.content.Context that get expect to get through the constructor:

    FibonacciService/src/com/marakana/android/fibonacciservice/IFibonacciServiceImpl.java:
    package com.marakana.android.fibonacciservice;

import android.content.Context;
…
public class IFibonacciServiceImpl extends IFibonacciService.Stub {
    …
    private final Context context;

    public IFibonacciServiceImpl(Context context) {
        this.context = context;
    }

    private long checkN(long n) {
        if (n > 10) {
            this.context.enforceCallingOrSelfPermission(
                    Manifest.permission.USE_SLOW_FIBONACCI_SERVICE, "Go away!");
        }
        return n;
    }
    …
    public long fibJR(long n) {
        …
        return FibLib.fibJR(this.checkN(n));
    }
    …
    public long fibNR(long n) {
        …
        return FibLib.fibNR(this.checkN(n));
    }
    …
}

  3. We have to update FibonacciService to invoke the new IFibonacciServiceImpl's constructor:

    FibonacciService/src/com/marakana/android/fibonacciservice/FibonacciService.java:
    …
public class FibonacciService extends Service {
    …
    @Override
    public void onCreate() {
        …
        this.service = new IFibonacciServiceImpl(super.getApplicationContext());
        …
    }
    …
}

  4. If we now re-run the FibonacciService and re-run the FibonacciClient for a recursive operation with n > 10, we will notice that the client will fail and adb logcat will show something like: 

    …
D/IFibonacciServiceImpl(  617): fib(15, RECURSIVE_NATIVE)
D/IFibonacciServiceImpl(  617): fibNR(15)
W/dalvikvm(  604): threadid=11: thread exiting with uncaught exception (group=0x409c01f8)
E/AndroidRuntime(  604): FATAL EXCEPTION: AsyncTask #1
E/AndroidRuntime(  604): java.lang.RuntimeException: An error occured while executing doInBackground()
…
E/AndroidRuntime(  604):    at java.lang.Thread.run(Thread.java:856)
E/AndroidRuntime(  604): Caused by: java.lang.SecurityException: Go away!: Neither user 10043 nor current process has com.marakana.android.fibonacciservice.USE_SLOW_FIBONACCI_SERVICE.
…

  5. Finally, we can allow FibonacciClient to melt our CPU and drain our battery by allowing it to use the USE_SLOW_FIBONACCI_SERVICE permission:

    FibonacciClient/AndroidManifest.xml:
    <?xml version="1.0" encoding="utf-8"?>
<manifest …>
    …
    <uses-permission android:name="com.marakana.android.fibonacciservice.USE_SLOW_FIBONACCI_SERVICE"/>
    …
</manifest>

  6. We can now observe that our client is again able to use recursive fibonacci operations even for n > 10

  7. In the Emulator, if we go to Home → Menu → Manage apps → Fibonacci Client → PERMISSIONS → Fibonacci Permissions, we should see both use fibonacci service and use slow fibonacci service operations permissions

   Lab: Custom Permissions

   ContentProvider URI Permissions

   Public vs. Private Components

   Intent Broadcast Permissions

   Pending Intents

   Encryption

Data encryption

Note In May 2011, Google was caught with their pants down. They passed authentication tokens from their Android client applications to their backend services, including contacts, calendars, and photos (picassa) over a plain-text (i.e. unencrypted) channel. This enabled potential attackers to get access and modify private content of users whose auth tokens were captured.
See http://money.cnn.com/2011/05/18/technology/android_security/?section=money_latest

Whole Disk Encryption

Caution Breaks during 3.0 to 3.1 OS upgrade. A 3.0-encrypted device had to be master-reset (i.e. all data on /data had to be wiped) on upgrading to 3.1.
Caution While Honeycomb’s whole-disk encryption based on dm-crypt is clearly a step in the right direction, it is far from being a NIST FIPS 140-2-certified solution, which requires two-factor encryption and is mandated for most of DOD applications.
Note Apple’s iOS 4 256-bit hardware encryption was cracked in May 2011 by ElcomSoft through a "simple" brute-force attack in as little as 30 mins using CPU and GPUs of modern host machines. See http://www.geek.com/articles/chips/apples-ios-4-hardware-encryption-has-been-cracked-20110525/ for more info.
Also, according to Nguyen from Symantec, iOS encryption key is stored on the device, but itself is not encrypted by the user’s master key. This means that if a potential attackers successfully jailbreak the device, they would be able to access the data without knowing the passcode.

VPN

Keystore and Keychain API

Application Encryption in Android 4.1

   Rooting an Android device

Controlling access to /system/bin/su with Superuser

Now, let take a look at some of the past Android exploits (Getting root the first time):

UDEV exploit (CVE-2009-1185)

ADB setuid exhaustion attack (CVE-2010-EASY)

Zimperlich attack against Zygote

Ashmem memory protection attack (CVE-2011-1149)

Buffer Overrun on vold exploit (CVE-2011-1823)

Linux Local Privilege Escalation via SUID /proc/pid/mem Write (CVE-2012-0056)

WebKit exploit

To Root or Not To Root?

Malware Rootkits

   Security of Memory

Most of the following comes from Android’s own documentation on Memory Management Security Enhancements.

Android 1.5+

Android 2.3+

Android 4.0+

Android 4.1+

Note See Exploit Mitigations in Android Jelly Bean 4.1 for more detailed explanation of the changes that went into 4.1

   Tap-Jacking on Android

   Android Device Administration

Device Administration Overview

The Android Device Administration API, introduced in Android 2.2, allows you to create security-aware applications that are useful in enterprise settings, such as:

You use the Device Administration API to write device admin applications that users install on their devices. The device admin application enforces desired security policies. Here’s how it works:

When enabled, in addition to enforcing security policies, the admin application can:

Device Administration Overview (cont.)

If a device contains multiple enabled admin applications, the strictest policy is enforced.

If users do not enable the device admin app, it remains on the device, but in an inactive state.

If a user fails to comply with the policies (for example, if a user sets a password that violates the guidelines), it is up to the application to decide how to handle this.

To uninstall an existing device admin application, users need to first deactivate the application as a device administrator.

Security Policies

An admin application may enforce security policies regarding the device’s screen lock PIN/password, including:

Additionally, a security policy can require device storage encryption as of Android 3.0 and disabling of camera as for Android 4.0.

The Device Administration Classes

The Device Administration API includes the following classes:

DeviceAdminReceiver

Base class for implementing a device administration component. This class provides a convenience for interpreting the raw intent actions that are sent by the system. Your Device Administration application must include a DeviceAdminReceiver subclass.

DevicePolicyManager

A class for managing policies enforced on a device. Most clients of this class must have published a DeviceAdminReceiver that the user has currently enabled. The DevicePolicyManager manages policies for one or more DeviceAdminReceiver instances.

DeviceAdminInfo

This class is used to specify metadata for a device administrator component.

Creating the Manifest

The manifest of your admin application must register your DeviceAdminReceiver as a <receiver>.

The <receiver> should set android:permission="android.permission.BIND_DEVICE_ADMIN" to ensure that only the system is allowed to interact with the broadcast receiver.

The <receiver> must have an <intent-filter> child element including one or more of the following <action>s, as defined in the DeviceAdminReceiver class:

ACTION_DEVICE_ADMIN_ENABLED

(Required) This is the primary action that a device administrator must implement to be allowed to manage a device. This is sent to a device administrator when the user enables it for administration.

ACTION_DEVICE_ADMIN_DISABLE_REQUESTED

Action sent to a device administrator when the user has requested to disable it, but before this has actually been done.

ACTION_DEVICE_ADMIN_DISABLED

Action sent to a device administrator when the user has disabled it.

ACTION_PASSWORD_CHANGED

Action sent to a device administrator when the user has changed the password of their device.

ACTION_PASSWORD_EXPIRING

Action periodically sent to a device administrator when the device password is expiring.

ACTION_PASSWORD_FAILED

Action sent to a device administrator when the user has failed at attempted to enter the password.

ACTION_PASSWORD_SUCCEEDED

Action sent to a device administrator when the user has successfully entered their password, after failing one or more times. 

<receiver android:name="MyDeviceAdminReceiver"
    android:permission="android.permission.BIND_DEVICE_ADMIN">
    <intent-filter>
        <action android:name="android.app.action.DEVICE_ADMIN_ENABLED" />
        <action android:name="android.app.action.ACTION_DEVICE_ADMIN_DISABLE_REQUESTED" />
        <action android:name="android.app.action.ACTION_DEVICE_ADMIN_DISABLED" />
    </intent-filter>
    <!-- ... -->
</receiver>

Creating the Manifest (cont.)

Your <receiver> element must also include a <meta-data> child element specifying an XML resource declaring the policies used by your admin application.

<meta-data android:name="android.app.device_admin"
           android:resource="@xml/device_admin_sample" />

An example XML resource requesting all policies would be:

<device-admin xmlns:android="http://schemas.android.com/apk/res/android">
    <uses-policies>
        <limit-password />
        <watch-login />
        <reset-password />
        <force-lock />
        <wipe-data />
        <expire-password />
        <encrypted-storage />
        <disable-camera />
    </uses-policies>
</device-admin>

Your application needs to list only those policies it actually uses.

The DeviceAdminReceiver Class

The DeviceAdminReceiver class defines a set of methods that you can override to handle the device administration events broadcast by the system:

void onEnabled(Context context, Intent intent)

Called after the administrator is first enabled, as a result of receiving ACTION_DEVICE_ADMIN_ENABLED. At this point you can use DevicePolicyManager to set your desired policies.

CharSequence onDisableRequested(Context context, Intent intent)

Called when the user has asked to disable the administrator, as a result of receiving ACTION_DEVICE_ADMIN_DISABLE_REQUESTED. You may return a warning message to display to the user before being disabled, or null for no message.

void onDisabled(Context context, Intent intent)

Called prior to the administrator being disabled, as a result of receiving ACTION_DEVICE_ADMIN_DISABLED. Upon return, you can no longer use the protected parts of the DevicePolicyManager API.

void onPasswordChanged(Context context, Intent intent)

Called after the user has changed their password, as a result of receiving ACTION_PASSWORD_CHANGED.

void onPasswordExpiring(Context context, Intent intent)

Called periodically when the password is about to expire or has expired, as a result of receiving ACTION_PASSWORD_EXPIRING. (API 11)

void onPasswordFailed(Context context, Intent intent)

Called after the user has failed at entering their current password, as a result of receiving ACTION_PASSWORD_FAILED.

void onPasswordSucceeded(Context context, Intent intent)

Called after the user has succeeded at entering their current password, as a result of receiving ACTION_PASSWORD_SUCCEEDED.

Testing Whether the Admin Application is Enabled

You can query the DevicePolicyManager to test if your admin application is enabled:

DevicePolicyManager devicePolicyManager
    = (DevicePolicyManager) getSystemService(Context.DEVICE_POLICY_SERVICE);
ComponentName deviceAdminComponentName
    = new ComponentName(this, MyDeviceAdminReceiver.class);
boolean isActive = devicePolicyManager.isAdminActive(deviceAdminComponentName);

You could then enable or disable features of your application depending on whether it is an active device administrator.

Enabling the Application

Your application must explicitly request the user to enable it for device administration. To do so:

  1. Create an implicit Intent with the DevicePolicyManager.ACTION_ADD_DEVICE_ADMIN action: 

    Intent intent = new Intent(DevicePolicyManager.ACTION_ADD_DEVICE_ADMIN);

  2. Add an extra identifying your DeviceAdminReceiver component: 

    ComponentName deviceAdminComponentName
    = new ComponentName(this, MyDeviceAdminReceiver.class);
intent.putExtra(DevicePolicyManager.EXTRA_DEVICE_ADMIN, deviceAdminComponentName);

  3. Optionally, provide an explanation as to why the user should activate the admin application: 

    intent.putExtra(DevicePolicyManager.EXTRA_ADD_EXPLANATION, "Your boss told you to do this");

  4. Use the Intent with startActivityForResult() to display the activation dialog: 

    startActivityForResult(intent, ACTIVATION_REQUEST);

  5. You can test for successful activation in your Activity’s onActivityResult() method: 

    @Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
    switch (requestCode) {
        case ACTIVATION_REQUEST:
            if (resultCode == Activity.RESULT_OK) {
                Log.i("DeviceAdminSample", "Administration enabled!");
            } else {
                Log.i("DeviceAdminSample", "Administration enable FAILED!");
            }
            return;
    }
    super.onActivityResult(requestCode, resultCode, data);
}

Setting Password Quality Policies

DevicePolicyManager includes APIs for setting and enforcing the device screen lock password policy.

Your application’s policy metadata resource must request the <limit-password /> policy to control password quality; otherwise these methods throw a security exception.

Setting Password Quality Policies, API 11

Beginning with Android 3.0, the DevicePolicyManager class includes methods that give you greater control over the contents of the password. Here are the methods for fine-tuning a password’s contents:

You can also set the password expiration timeout, and prevent users from reusing the last n unique passwords:

Additionally, Android 3.0 introduced support for a policy requiring the user to encrypt the device, which you can set with:

Your application’s policy metadata resource must request the <limit-password /> policy to control password quality; otherwise these methods throw a security exception.

Similarly, it must request the <expire-password /> and <encrypted-storage /> policies to control those features without throwing a security exception.

Setting the Device Password

You can test if the current device password meets the quality requirements by calling DevicePolicyManager.isActivePasswordSufficient(), which returns a boolean result.

If necessary, you can start an activity prompting the user to set a password as follows:

Intent intent = new Intent(DevicePolicyManager.ACTION_SET_NEW_PASSWORD);
startActivity(intent);

Your application can also perform a password reset on the device using DevicePolicyManager.resetPassword(). This can be useful if your application is designed to support remote administration, with a new password being provided from a central administration system.

Your application’s policy metadata resource must request the <reset-password /> policy to reset the password; otherwise resetPassword() throws a security exception.

Locking and Wiping the Device

Your application can lock the device programmatically using DevicePolicyManager.lockNow().

You can wipe the user data of the device, performing a factory reset, using DevicePolicyManager.wipeData().

Additionally, you can set the maximum number of allowed failed password attempts before the device is wiped automatically by calling DevicePolicyManager.setMaximumFailedPasswordsForWipe()

Your application’s policy metadata resource must request the <wipe-data /> policy to wipe the data either explicitly or set the maximum failed passwords for wipe; otherwise a security exception is thrown. setMaximumFailedPasswordsForWipe() also requires the <watch-login /> policy.

The lockNow() method requires your application to request the <force-lock /> policy to avoid throwing a security exception.

Device Administration Demo

In this example app, you will see how to write an application that requests to device administration privileges, and once it gets them, allows user to lock or reset the device.

We are going to look at the following files:

The source code for this project is available at https://marakana.com/static/courseware/android/DevicePolicyDemo.zip

Android Manifest File

This is where we register our device administration receiver component. It appears as another receiver declaration.

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.marakana.android.devicepolicydemo"
    android:versionCode="1"
    android:versionName="1.0" >

    <uses-sdk android:minSdkVersion="10" />

    <application
        android:icon="@drawable/ic_launcher"
        android:label="@string/app_name" >
        <activity
            android:name=".DevicePolicyDemoActivity"
            android:label="@string/app_name" >
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>

        <!-- This is where we register our receiver -->
        <receiver
            android:name=".DemoDeviceAdminReceiver"
            android:permission="android.permission.BIND_DEVICE_ADMIN" >
            <intent-filter>

                <!-- This action is required -->
                <action android:name="android.app.action.DEVICE_ADMIN_ENABLED" />
            </intent-filter>

            <!-- This is required this receiver to become device admin component. -->
            <meta-data
                android:name="android.app.device_admin"
                android:resource="@xml/device_admin_sample" />
        </receiver>
    </application>

</manifest>

Notice that <receiver> element now includes required android:permission="android.permission.BIND_DEVICE_ADMIN" permission declaration.

We also must include the appropriate intent action filter android.app.action.DEVICE_ADMIN_ENABLED as well as the <meta-data/> element that specifies that this receiver users @xml/device_admin_sample resource, which we’ll look at next.

XML Resource File

This XML resource file, referenced from AndroidManifest.xml specifies what policies we are interested in.

<device-admin xmlns:android="http://schemas.android.com/apk/res/android">
        <uses-policies>
                <limit-password />
                <watch-login />
                <reset-password />
                <force-lock />
                <wipe-data />
                <expire-password />
                <encrypted-storage />
        </uses-policies>
</device-admin>

In this example, we ask for most of the available policies merely to illustrate what is available. In a real-world example, you should only ask for policies that you really require.

Device Admin Receiver Component

This is the main device administration component. It is basically a specialized BroadcastReceiver class that implements some callbacks specific to device administration.

package com.marakana.android.devicepolicydemo;

import android.app.admin.DeviceAdminReceiver;
import android.content.Context;
import android.content.Intent;
import android.util.Log;
import android.widget.Toast;

/**
 * This is the component that is responsible for actual device administration.
 * It becomes the receiver when a policy is applied. It is important that we
 * subclass DeviceAdminReceiver class here and to implement its only required
 * method onEnabled().
 */
public class DemoDeviceAdminReceiver extends DeviceAdminReceiver {
    static final String TAG = "DemoDeviceAdminReceiver";

    /** Called when this application is approved to be a device administrator. */
    @Override
    public void onEnabled(Context context, Intent intent) {
        super.onEnabled(context, intent);
        Toast.makeText(context, R.string.device_admin_enabled,
                Toast.LENGTH_LONG).show();
        Log.d(TAG, "onEnabled");
    }

    /** Called when this application is no longer the device administrator. */
    @Override
    public void onDisabled(Context context, Intent intent) {
        super.onDisabled(context, intent);
        Toast.makeText(context, R.string.device_admin_disabled,
                Toast.LENGTH_LONG).show();
        Log.d(TAG, "onDisabled");
    }

    @Override
    public void onPasswordChanged(Context context, Intent intent) {
        super.onPasswordChanged(context, intent);
        Log.d(TAG, "onPasswordChanged");
    }

    @Override
    public void onPasswordFailed(Context context, Intent intent) {
        super.onPasswordFailed(context, intent);
        Log.d(TAG, "onPasswordFailed");
    }

    @Override
    public void onPasswordSucceeded(Context context, Intent intent) {
        super.onPasswordSucceeded(context, intent);
        Log.d(TAG, "onPasswordSucceeded");
    }

}

Notice that we subclass DeviceAdminReceiver class. This is the required for this component to be able to receive policy notifications.

We also must implement the required onEnabled() method that is called when the policy administration is first enabled.

We don’t really do much here other than log what happened to visually illustrate the execution of this code.

Activity

The activity acts as our demo client in this case. The significant methods are onClick(), onCheckedChanged() and onActivityResult().

package com.marakana.android.devicepolicydemo;

import android.app.Activity;
import android.app.admin.DevicePolicyManager;
import android.content.ComponentName;
import android.content.Context;
import android.content.Intent;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.widget.CompoundButton;
import android.widget.CompoundButton.OnCheckedChangeListener;
import android.widget.Toast;
import android.widget.ToggleButton;

public class DevicePolicyDemoActivity extends Activity implements
        OnCheckedChangeListener {
    static final String TAG = "DevicePolicyDemoActivity";
    static final int ACTIVATION_REQUEST = 47; // identifies our request id
    DevicePolicyManager devicePolicyManager;
    ComponentName demoDeviceAdmin;
    ToggleButton toggleButton;

    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);

        toggleButton = (ToggleButton) super
                .findViewById(R.id.toggle_device_admin);
        toggleButton.setOnCheckedChangeListener(this);

        // Initialize Device Policy Manager service and our receiver class
        devicePolicyManager = (DevicePolicyManager) getSystemService(Context.DEVICE_POLICY_SERVICE);
        demoDeviceAdmin = new ComponentName(this, DemoDeviceAdminReceiver.class);
    }

    /**
     * Called when a button is clicked on. We have Lock Device and Reset Device
     * buttons that could invoke this method.
     */
    public void onClick(View v) {
        switch (v.getId()) {
        case R.id.button_lock_device:
            // We lock the screen
            Toast.makeText(this, "Locking device...", Toast.LENGTH_LONG).show();
            Log.d(TAG, "Locking device now");
            devicePolicyManager.lockNow();
            break;
        case R.id.button_reset_device:
            // We reset the device - this will erase entire /data partition!
            Toast.makeText(this, "Locking device...", Toast.LENGTH_LONG).show();
            Log.d(TAG,
                    "RESETing device now - all user data will be ERASED to factory settings");
            devicePolicyManager.wipeData(ACTIVATION_REQUEST);
            break;
        }
    }

    /**
     * Called when the state of toggle button changes. In this case, we send an
     * intent to activate the device policy administration.
     */
    @Override
    public void onCheckedChanged(CompoundButton button, boolean isChecked) {
        if (isChecked) {
            // Activate device administration
            Intent intent = new Intent(
                    DevicePolicyManager.ACTION_ADD_DEVICE_ADMIN);
            intent.putExtra(DevicePolicyManager.EXTRA_DEVICE_ADMIN,
                    demoDeviceAdmin);
            intent.putExtra(DevicePolicyManager.EXTRA_ADD_EXPLANATION,
                    "Your boss told you to do this");
            startActivityForResult(intent, ACTIVATION_REQUEST);
        }
        Log.d(TAG, "onCheckedChanged to: " + isChecked);
    }

    /**
     * Called when startActivityForResult() call is completed. The result of
     * activation could be success of failure, mostly depending on user okaying
     * this app's request to administer the device.
     */
    @Override
    protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        switch (requestCode) {
        case ACTIVATION_REQUEST:
            if (resultCode == Activity.RESULT_OK) {
                Log.i(TAG, "Administration enabled!");
                toggleButton.setChecked(true);
            } else {
                Log.i(TAG, "Administration enable FAILED!");
                toggleButton.setChecked(false);
            }
            return;
        }
        super.onActivityResult(requestCode, resultCode, data);
    }

}

onCheckedChanged() is invoked when the toggle button changes state. It sends an intent requesting that this application be granted device administration permissions on this device. User has to allow this request. The result of user’s action is then passed to onActivityResult() method.

onClick() method processes button clicks for lock and reset buttons. Note that its calls to DevicePolicyManager will lock and wipe out the device user data partition, respectively.

Lab (Device Administration)

Device-admin-enable an existing Android app:

  1. Download http://marakana.com/static/courseware/android/SecureNote.zip

  2. Expand into your (Eclipse) workspace

  3. Import into Eclipse

  4. Using Device Admin APIs

    1. Require that the screen-lock password be set (of at least 6 characters with at least one digit)

    2. Automatically wipe the device after 5 invalid login attempts

  5. Check against the solution available at http://marakana.com/static/courseware/android/DeviceAdministeredSecureNote.zip

   Malware

Lessons From The Field

Detection

Removal

   SE Android (SE-Linux on Android)

   Other Security Concerns

   Review Questions

  1. What is the basic philosophy of the Android security model?

  2. What does Android do to implement this philosophy?

  3. How and when are permissions used, granted, and enforced?

  4. What is a confused deputy attack?

  5. What is a collusion attack?

  6. Which part of Android is ultimately responsible for providing the system security?

  7. How does native code differ from Java code when it comes to its security context?

  8. Why do we sign applications?

  9. What are the four platform keys?

  10. What is the relationship between apps and Linux user IDs?

  11. When are the user IDs assigned to apps?

  12. Where are the user ID assignments recorded?

  13. What is the relationship between application user IDs and its file-system resources?

  14. Where is the home of an app with the package name com.foo.bar?

  15. How do apps share a user IDs?

  16. How do apps share a process? Provide at least one use-case for this.

  17. Name at least five (file) system volumes (think mount points).

  18. What is the purpose of /data/dalvik-cache/?

  19. What is the purpose of run-as command?

  20. How do you use permissions?

  21. What happens if you don’t use a permission but you attempt to use a restricted API?

  22. How are the permissions enforced?

  23. How do bound services enforce their permissions? What makes that possible?

  24. What are the attributes of the <permission … /> tag?

  25. What are the possible values for protectionLevel attribute?

  26. What does protectedLevel=signatureOrSystem mean?

  27. What is the value of grouping permissions?

  28. What is the significance of android:grantUriPermission attribute on content providers?

  29. What is the significance android:exported attribute on Android components?

  30. How do you protected unauthorized applications from receiving your broadcasts (intents)?

  31. What’s the worry with pending intents?

  32. What is the purpose of data encryption and when is it useful?

  33. What is CBC?

  34. What is IV?

  35. What is the "whole disk encryption"?

  36. What can we do with VPN on ICS?

  37. How can we manage keys on ICS?

  38. What are the dangers of rooting?

  39. How do we get root?

  40. How can we "test" for root?

  41. What is ALSR? And what is its purpose?

  42. What is NX?

  43. What is tap jacking?

  44. What is the purpose of Device Admin API? What can you do with it?

  45. What are the ways we can identify malware? When can we do this?

  46. What is DAC? What is MAC? And what can we do with SE-Android?

Building Android From Source

   Why Build Android From Source?

   Setting up the Build Environment

   Downloading the Source Tree

   Android Source Code Structure

   Android Build System

   Initializing the Build Environment

   Choosing the Build Target

Note Additional CodeName-BuildType combinations may be available based on what build/envsetup.sh finds (usually in the device/ folder)
Note Building for real hardware usually requires that we get proprietary binaries (mostly user-space HAL). For Nexus and other Google-supported devices, we can go to http://code.google.com/android/nexus/drivers.html, which allows us to download scripts, which in turn extract binaries from the connected devices (via adb pull) into vendor/ directory tree structure.

   Compiling Android

Makefile targets

Android build system Makefile (actually build/core/main.mk) includes some of the following targets:

Make targets Description

droid

The default target (build the full system)

clean

Equivalent to rm -rf out/ (same as make clobber)

installclean

Deletes all of the files that change between different build types, like make user vs. make sdk

dataclean

Delete files in the staging and emulator data partitions: data/*, data-qemu/*, and userdata-qemu.img

snod

Quickly rebuild the system image from built packages

offline-sdk-docs

Generate the HTML for the developer SDK docs

doc-comment-check-docs

Check HTML doc links and validity, without generating HTML

libandroid_runtime

All the JNI framework stuff

framework

All the java framework stuff

services

The system server (Java) and friends

sdk

Build the Android SDK (tools)

help

Display a help message listing some of these targets

modules

Display a list of modules that can be built (where each module name is specified by LOCAL_MODULE)

<module-name>

Make just a specific module (same as cd module/dir && mm)

clean-<module-name>

Clean just a specific module

otacerts

OTA keys that are used to verify OTA packages

…

Etc.

   Examining the Built Images

   Running Custom Android Build on Emulator

   Running Custom Android Build on Real Hardware

   Building the Linux Kernel

Note It is easiest to build the Linux kernel on a Linux OS. While other host OSs can also be used, they are not trivial to setup.

   Getting the Kernel

$ git clone https://android.googlesource.com/kernel/common.git
$ git clone https://android.googlesource.com/kernel/goldfish.git
$ git clone https://android.googlesource.com/kernel/msm.git
$ git clone https://android.googlesource.com/kernel/omap.git
$ git clone https://android.googlesource.com/kernel/samsung.git
$ git clone https://android.googlesource.com/kernel/tegra.git

Building Kernel for the Emulator (Goldfish)

  1. Start the emulator

  2. Get the existing kernel version from /proc/version (since uname does not exist on Android) 

    $ adb shell cat /proc/version
Linux version 2.6.29-g46b05b2 (vchtchetkine@vc-irv.irv.corp.google.com) (gcc version 4.4.3 (GCC) ) #28 Thu Nov 17 06:39:36 PST 2011

  3. Get the corresponding kernel version (here, we are getting the kernel for goldfish, the emulator) 

    $ git clone https://android.googlesource.com/kernel/goldfish.git
$ cd goldfish/
$ git branch -a
$ git checkout -t remotes/origin/android-goldfish-2.6.29
    Note Alternatively, we could directly clone the goldfish 2.6.29 branch
    $ git clone https://android.googlesource.com/kernel/goldfish.git -b android-goldfish-2.6.29

  4. Specify the target architecture and cross compiler 

    $ export ARCH=arm
$ export CROSS_COMPILE=/path/to/android-src/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-
    Caution Android’s own build system uses ARCH and CROSS_COMPILE env vars, so these need to be cleared before building AOSP.
    As an alternative, these can be also passed directly to the make commands: $ make … ARCH=… CROSS_COMPILE=…

  5. Get the existing kernel configuration file (with all of the Android/emulator specific options) by pulling it from the running emulator: 

    $ adb pull /proc/config.gz .
$ gunzip config.gz
$ mv config .config
    Note As an alternative, we could generate the Goldfish .config file (even without the emulator) by running:
    $ make goldfish_armv7_defconfig ARCH=arm

  6. You can optionally take a look at the existing configuration options and make changes as desired 

    $ make menuconfig ARCH=arm
    • For example, you could set your General setup → Local version - append to kernel release to something like -marakana-example

  7. Now we are ready to compile 

    $ make

  8. The resulting kernel will be compressed to arch/arm/boot/zImage

  9. Run the emulator with our new kernel 

    $ out/host/linux-x86/bin/emulator -kernel /path/to/common/arch/arm/boot/zImage

  10. And the result is

    screens/CustomKernel.png

   Review Questions

  1. What is repo?

  2. What Android’s build system based on?

  3. What is the first thing you need to do before you can build Android (after you get the source)?

  4. What is the unit of build?

  5. What are the different build types and how do they differ?

  6. What is the purpose of ccache?

  7. What is the end result of a build?

  8. What’s inside recovery.img?

  9. What is the purpose of fastboot?

  10. What is the purpose of CROSS_COMPILE variable and what do you set it to when compiling the kernel?

Android Startup

images/AndroidProcessSnapshot.svg

   Bootloading the Kernel

  1. On power-up, CPU is uninitialized - wait for stable power

  2. Execute Boot ROM (hardwired into CPU)

    1. Locate the first-stage boot loader

    2. Load the first-stage boot loader into internal RAM

    3. Jump to first-stage boot loader’s memory location to execute it

  3. First-stage boot loader runs

    1. Detect and initialize external RAM

    2. Locate the second-stage boot loader

    3. Load the second-stage boot loader into external RAM

    4. Jump to the second-stage boot loader’s memory location to execute it

  4. Second-stage boot loader runs

    1. Setup file systems (typically on Flash media)

    2. Optionally setup display, network, additional memory, and other devices

    3. Enable additional CPU features

    4. Enable low-level memory protection

    5. Optionally load security protections (e.g. ROM validation code)

    6. Locate Linux Kernel

    7. Load Linux Kernel into RAM

    8. Place Linux Kernel boot parameters into memory so that kernel knows what to run upon startup

    9. Jump to Linux Kernel memory address to run it

  5. Linux Kernel runs

    1. Build a table in RAM describing the layout of the physical memory

    2. Initialize and setup input devices

    3. Initialize and setup disk (typically MTD) controllers and map available block devices in RAM

    4. Initialize Advanced Power Management (APM) support

    5. Initialize interrupt handlers: Interrupt Descriptor Table (IDT), Global Descriptor Table (GDT), and Programmable Interrupt Controllers (PIC)

    6. Reset the floating-point unit (FPU)

    7. Switch from real to protected mode (i.e. enable memory protection)

    8. Initialize segmentation registers and a provisional stack

    9. Zero uninitialized memory

    10. Decompress the kernel image

    11. Initialize provisional kernel page tables and enable paging

    12. Setup kernel mode stack for process 0

    13. Fill the IDT with null interrupt handlers

    14. Initialize the first page frame with system parameters

    15. Identify the CPU model

    16. Initialize registers with the addresses of the GDT and IDT

    17. Initialize and start the kernel

      1. Scheduler

      2. Memory zones

      3. Buddy system allocator

      4. IDT

      5. SoftIRQs

      6. Date and Time

      7. Slab allocator

      8. …

    18. Create process 1 (/init) and run it

   Android’s init Startup

images/StartupWalkthru.svg

   Zygote Startup

  1. Zygote starts from /init.rc 

    service zygote /system/bin/app_process -Xzygote /system/bin --zygote --start-system-server
    socket zygote stream 666
    onrestart write /sys/android_power/request_state wake
    onrestart write /sys/power/state on
    onrestart restart media
    onrestart restart netd

  2. This translates to frameworks/base/cmds/app_process/app_main.cpp:main()

  3. The command app_process then launches frameworks/base/core/java/com/android/internal/os/ZygoteInit.java:main() in a Dalvik VM via frameworks/base/core/jni/AndroidRuntime.cpp:start()

  4. ZygoteInit.main() then

    1. Registers for zygote socket

    2. Pre-loads classes defined in frameworks/base/preloaded-classes (1800+)

    3. Pre-loads resources preloaded_drawables and preloaded_color_state_lists from frameworks/base/core/res/res/values/arrays.xml

    4. Runs garbage collector (to clean the memory as much as possible)

    5. Forks itself to start system_server

    6. Starts listening for requests to fork itself for other apps

   System Server Startup

  1. When Zygote forks itself to launch the system_server process (in ZygoteInit.java:startSystemServer()), it executes frameworks/base/services/java/com/android/server/SystemServer:java.main()

  2. The SystemServer:java.main() method loads android_servers JNI lib from frameworks/base/services/jni and invokes init1() native method

  3. Before init1() runs, the JNI loader first runs frameworks/base/services/jni/onload.cpp:JNI_OnLoad(), which registers native services - to be used as JNI counterparts to Java-based service manager loaded later

  4. Now frameworks/base/services/jni/com_android_server_SystemServer.cpp:init1() is invoked, which simply wraps a call to frameworks/base/cmds/system_server/library/system_init.cpp:system_init()

  5. The system_init.cpp:system_init() function

    1. First starts native services (some optionally):

      1. frameworks/base/services/surfaceflinger/SurfaceFlinger.cpp

      2. frameworks/base/services/sensorservice/SensorService.cpp

      3. frameworks/base/services/audioflinger/AudioFlinger.cpp

      4. frameworks/base/media/libmediaplayerservice/MediaPlayerService.cpp

      5. frameworks/base/camera/libcameraservice/CameraService.cpp

      6. frameworks/base/services/audioflinger/AudioPolicyService.cpp

    2. Then goes back to frameworks/base/services/java/com/android/server/SystemServer.java:init2(), again via frameworks/base/core/jni/AndroidRuntime.cpp:start() JNI call

  6. The SystemServer.java:init2() method then starts Java service managers in a separate thread (ServerThread), readies them, and registers each one with frameworks/base/core/java/android/os/ServiceManager:addService() (which in turn delegates to to ServiceManagerNative.java, which effectively talks to servicemanager daemon previously started by init)

    1. frameworks/base/services/java/com/android/server/PowerManagerService.java

    2. frameworks/base/services/java/com/android/server/am/ActivityManagerService.java

    3. frameworks/base/services/java/com/android/server/TelephonyRegistry.java

    4. frameworks/base/services/java/com/android/server/PackageManagerService.java

    5. frameworks/base/services/java/com/android/server/BatteryService.java

    6. frameworks/base/services/java/com/android/server/VibratorService.java

    7. etc

  7. Finally frameworks/base/services/java/com/android/server/am/ActivityManagerService.java:finishBooting() sets sys.boot_completed=1 and sends out

    1. a broadcast intent with android.intent.action.PRE_BOOT_COMPLETED action (to give apps a chance to reach to boot upgrades)

    2. an activity intent with android.intent.category.HOME category to launch the Home (or Launcher) application

    3. a broadcast intent with android.intent.action.BOOT_COMPLETED action, which launches applications subscribed to this intent (while using android.permission.RECEIVE_BOOT_COMPLETED)

   Review Questions

  1. What is the purpose of the second-stage bootloader?

  2. What is the first thing that runs (in the user-space) after the kernel is initialized?

  3. Where does this process (with PID==1) get its initialization instructions from?

  4. What are actions (of process with PID==1)?

  5. Name at least three triggers and three commands for actions.

  6. What are services (of process with PID==1)?

  7. How are services launched?

  8. What’s the difference between critical and non-critical services?

  9. Name at least five services started by process with PID==1.

  10. What is the purpose of ueventd process?

  11. What is the purpose of zygote process and what does it do upon initialization?

  12. What is the purpose of system_server process and what does it do upon initialization?

  13. Who is responsible for launching the Home application and how is it done?

  14. How do applications that want to start on boot get launched?

Android Services

The objective of this module is to explain the inter-workings of various Android services. There are close to sixty various services in ICS release. In this module, we’ve hand-picked some of the more common ones. By the end of the module, you should start to understand some common traits of Android service architecture, such as use of the Binder for inter-process communication, and use of JNI for Java-C interaction.

Android services are the key to exposing lower level functionality of the hardware and the Linux kernel to the high level Android apps. Understanding how they work creates the opportunity to customize and extend their behavior, or add another service altogether.

$ adb shell service list
Found 56 services:
0   phone: [com.android.internal.telephony.ITelephony]
1   iphonesubinfo: [com.android.internal.telephony.IPhoneSubInfo]
2   simphonebook: [com.android.internal.telephony.IIccPhoneBook]
3   isms: [com.android.internal.telephony.ISms]
4   samplingprofiler: []
5   diskstats: []
6   appwidget: [com.android.internal.appwidget.IAppWidgetService]
7   backup: [android.app.backup.IBackupManager]
8   uimode: [android.app.IUiModeManager]
9   usb: [android.hardware.usb.IUsbManager]
10  audio: [android.media.IAudioService]
11  wallpaper: [android.app.IWallpaperManager]
12  dropbox: [com.android.internal.os.IDropBoxManagerService]
13  search: [android.app.ISearchManager]
14  country_detector: [android.location.ICountryDetector]
15  location: [android.location.ILocationManager]
16  devicestoragemonitor: []
17  notification: [android.app.INotificationManager]
18  mount: [IMountService]
19  throttle: [android.net.IThrottleManager]
20  connectivity: [android.net.IConnectivityManager]
21  wifi: [android.net.wifi.IWifiManager]
22  wifip2p: [android.net.wifi.p2p.IWifiP2pManager]
23  netpolicy: [android.net.INetworkPolicyManager]
24  netstats: [android.net.INetworkStatsService]
25  textservices: [com.android.internal.textservice.ITextServicesManager]
26  network_management: [android.os.INetworkManagementService]
27  clipboard: [android.content.IClipboard]
28  statusbar: [com.android.internal.statusbar.IStatusBarService]
29  device_policy: [android.app.admin.IDevicePolicyManager]
30  accessibility: [android.view.accessibility.IAccessibilityManager]
31  input_method: [com.android.internal.view.IInputMethodManager]
32  window: [android.view.IWindowManager]
33  alarm: [android.app.IAlarmManager]
34  vibrator: [android.os.IVibratorService]
35  battery: []
36  hardware: [android.os.IHardwareService]
37  content: [android.content.IContentService]
38  account: [android.accounts.IAccountManager]
39  permission: [android.os.IPermissionController]
40  cpuinfo: []
41  gfxinfo: []
42  meminfo: []
43  activity: [android.app.IActivityManager]
44  package: [android.content.pm.IPackageManager]
45  telephony.registry: [com.android.internal.telephony.ITelephonyRegistry]
46  usagestats: [com.android.internal.app.IUsageStats]
47  batteryinfo: [com.android.internal.app.IBatteryStats]
48  power: [android.os.IPowerManager]
49  entropy: []
50  sensorservice: [android.gui.SensorServer]
51  media.audio_policy: [android.media.IAudioPolicyService]
52  media.camera: [android.hardware.IAudioPolicyServiceService]
53  media.player: [android.media.IMediaPlayerService]
54  media.audio_flinger: [android.media.IAudioFlinger]
55  SurfaceFlinger: [android.ui.ISurfaceComposer]

   Vibrator

images/VibratorArchitecture.svg
Vibrator Service Architecture

   Power Service

images/PowerManagerServiceArchitecture.svg
Power Service Architecture

Links

   Alarm Service

images/AlarmManagerServiceArchitecture.svg
Alarm Service Architecture

   Package Service

images/PackageManagerServiceArchitecture.svg
Package Service Architecture

   WiFi Service

Wifi Service exposes wifi functionality of the underlying system to the application layer via WifiManager class. The key differentiation between wifi stack and some other ones is that the wifi stack primarily uses the wpa_supplicant to manage the Wifi driver.

images/WifiArchitecture.svg
Wifi Service Architecture

Links

   Location Service

images/LocationManagerServiceArchitecture.svg
Location Service Architecture

   Audio Service

images/AudioPlaybackServiceArchitecture.svg
Audio Architecture (playing audio)

images/AudioManagerServiceArchitecture.svg

Links

   Android Media Framework

Introduction

The media framework are the APIs and libraries used for controlling playback and recording of video/audio. Since everything involving video and audio require a lot of computing power, mobile devices usually use a lot of special-purpose hardware for this, compared to desktop computers where there normally is enough raw power to run most/everything in software. Full-software solutions normally are more flexible and portable, but obviously require more processor power (which also might give a higher power consumption).

Normally only the highest level APIs are public, to allow for flexibility in the implementation internally.

Typical stack of function calls

  1. java: android.media.MediaPlayer
    frameworks/base/media/java/android/media/MediaPlayer.java

  2. JNI: frameworks/base/media/jni/android_media_MediaPlayer.cpp
    Quite straight mapping of java native functions to the C++ MediaPlayer class

  3. C++: MediaPlayer
    frameworks/base/media/libmedia/mediaplayer.cpp

  4. IMediaPlayer, IMediaPlayerService
    frameworks/base/include/media/IMediaPlayer.h
    frameworks/base/include/media/IMediaPlayerService.h

  5. BpMediaPlayer: Binder client/proxy interface
    frameworks/base/media/libmedia/IMediaPlayer.cpp

  6. BnMediaPlayer: Binder implementation interface (running in the media server)

  7. BnMediaPlayer implemented by MediaPlayerService::Client
    frameworks/base/libmediaplayservice/MediaPlayerService.cpp

  8. MediaPlayerService backed by different implementations:

    • PVPlayer (old OpenCORE based player, phased out in gingerbread, removed later)
    • StagefrightPlayer
      frameworks/base/media/libmediaplayerservice/StagefrightPlayer.cpp
      • AwesomePlayer
        frameworks/base/media/libstagefright/AwesomePlayer.cpp
      • OMXCodec
        frameworks/base/media/libstagefright/OMXCodec.cpp
        Pull based stagefright element, wrapping IOMX
      • AudioPlayer
        frameworks/base/media/libstagefright/AudioPlayer.cpp
    • NuPlayer (new in ICS/Honeycomb only for HTTP Live Streaming)
      frameworks/base/media/libmediaplayerservice/nuplayer/NuPlayerDriver.cpp
      • ACodec
        frameworks/base/media/libstagefright/ACodec.cpp message passing based frontend on top of IOMX

  9. MediaPlayerService::AudioOutput
    frameworks/base/libmediaplayservice/MediaPlayerService.cpp

  10. AudioTrack
    frameworks/base/media/libmedia/AudioTrack.cpp

  11. IAudioTrack

    • BpAudioTrack, BnAudioTrack

  12. AudioFlinger::TrackHandle
    frameworks/base/services/audioflinger/AudioFlinger.cpp

The actual decoding/encoding in the OMXCodec and ACodec class is implemented via the IOMX interface:

The applications may also use OpenSL ES for playing back audio this internally uses OMXCodec from stagefright for decoding of the audio. (Code for this is in system/media/opensles/libopensles/android SfPlayer.cpp in gingerbread, in system/media/wilhelm/src/android/android AudioSfDecoder.cpp in ICS.) In ICS, there’s also a an OpenMAX AL API that can play back video, using the IMediaPlayerService interface.

The media server process

Most of the APIs above in section 2 can either run in the calling application processes, or in the media server. Each of the Binder APIs (IAPI/BpAPI/BnAPI, such as IOMX, BpOMX, BnOMX) can proxy calls into the media server. The caller only gets an instance of the generic interface, e.g. IMediaPlayerm and does not know whether this is a proxy for calling the same interface in another process (BpOMX) or the actual implementation running within the same process (BnOMX).

Thanks to this, the whole concept of some parts of the API running within a separate process is mostly transparent when working with the APIs.

Stagefright

Intro

Stagefright is the library containing most of the implementation of the media framework.

Stagefright is a relatively recent development within android, replacing OpenCORE. It first appeared in Eclair, was made default player for all video (except for RTSP streaming which was still handled by OpenCORE) in Froyo, and in Gingerbread it had gained RTSP streaming support and was mature enough to be used for video recording, too, so OpenCORE was removed in Gingerbread. Since then, it has still evolved a bit.

Stagefright is mainly a generic pipeline kind of framework, similar to GStreamer, but much smaller and simpler. (It is purpose-written from scratch to fill the exact needs of Android, while OpenCORE was an already existing framework that PacketVideo provided at the time. Stagefright is a few orders of magnitude smaller than OpenCORE while still fulfilling all the needs Android has.)

Example

Being a pipeline framework means that it consists of individual elements that form a pipeline, where each element either can produce data itself, or take input data from another element. By chaining such elements together, one can produce a pipeline that does the intended thing. E.g., for playing back video from a mp4 file on the phone, the following elements are hooked up together:

  • `FileSource `
  • MPEG4Extractor
    One MPEG4Source for each track (audio/video) in the file
  • OMXCodec (one for each track, connected to the MPEG4Source)
  • AudioPlayer

The AwesomePlayer creates the right MediaExtractor in finishSetDataSource_1, setting up the tracks in setDataSource l, and later creates the OMXCodec object for each of the tracks. The audio OMXCodec object is given as source object to AudioPlayer, which then reads data from it.

GStreamer has the same element/pipeline design, but is much more flexible supporting a number of modes of operation and a lot more elements, while Stagefright is simpler and smaller, since it only does exactly what is needed in Android.

Pipeline

The pipeline in Stagefright is pull-based, meaning that each element deriving from the base class MediaSource implements a method read(), which when called will block until the MediaSource element has produced one MediaBuffer which is returned. When the MediaSource subclass object is created, it is normally given an upstream MediaSource object to read from, e.g. for OMXCodec, the MediaSource is given as parameter to OMXCodec::Create().

For OMXCodec, the read() method internally reads a packet of compressed data from the source (which might e.g. be MPEG4Source within MPEG4Extractor in the case above), passes the packet to the actual OpenMAX codec for decoding, and when the decoded raw audio data is available, it is returned by the read() method.

Similarly, the MPEG4Source object will block in the read() method until it has read one packet from the source file, which is then returned to the caller. With this design, elements can be more or less arbitrarily connected, as long as one element can handle the data type that the input element produces.

For pipelines, Stagefright contains these kinds of element implementations:

  • Sources for reading data from a file, or reading raw data from capture devices (e.g. CameraSource)
  • Extractors, for interpreting different file formats (e.g. AACExtractor, AMRExtractor, AVIExtractor, MP3Extractor, MPEG4Extractor, OggExtractor, WAVExtractor)
  • OMXCodec, for decoding/encoding data • Writers, for producing files of different formats (e.g. AACWriter, AMR-Writer, MPEG2TSWriter, MPEG4Writer)
Other components

In Honeycomb/Ice Cream Sandwich, Stagefright has evolved further, with a second mode of operation in addition to the plain pipeline based system. Now, the ACodec class encapsulates an OpenMAX codec with a different kind of abstraction. This class isn’t to be used as a strict pipeline element, thus, it doesn’t implement the MediaSource interface. Instead, messages are passed to ACodec (which implements AHandler, for receiving messages) with buffers to pass to the OpenMAX codec. This is more similar to the behavior model of the actual OpenMAX codecs themselves.

In addition to the elements, Stagefright contains higher level classes such as AwesomePlayer that coordinates playback of a video file, other utility classes such as color converters, wrapping of OpenMAX codecs (providing a Binder C++ interface around OpenMAX, giving it the possibility to use the codecs from a different process even if the codecs themselves run within the media server), and a number of software implementations of codecs.

images/MediaFrameworkComponents.svg
An illustration of the general pipeline structure.
Note FileSource, MPEG4Source and OMXCodec all implement the MediaSource interface. MPEG4Extractor, OMXCodec, AudioPlayer, MediaWriter all use the MediaSource interface for reading their input from the preceding element, of which they don’t need to know anything else than what’s in MediaSource.
images/MediaFrameworkPipeline.svg
An example of a pipeline decoding audio and video from an MP4 file.
images/MediaFrameworkArchitecture.svg
A rough visualization of the split between the application process and media server.
images/MediaFrameworkIOMX.svg
Details of the IOMX interface.

OpenMAX IL 5.1 Overview

OpenMAX IL in Stagefright

Stagefright provides wrapping around OMX core implementations. The main client api is called IOMX (frameworks/base/include/media/IOMX.h), which is a Binder interface with a proxy (BpOMX) and backend (BnOMX), allowing the caller and implementation to reside in separate processes. BnOMX is implemented by the OMX class in frameworks/base/media/libstagefright/omx/OMX.cpp, which uses the OMXMaster class in frameworks/base/media/libstagefright/omx/OMXMaster.cpp for querying and instantiating components from a number of different implementation sources. OMXMaster loads one or more plugins (OMXPluginBase, defined in frameworks/base/include/media/libstagefright/OMXPluginBase.h) that are analogous to normal OMX IL cores.

OMXMaster loads a shared library named libstagefrighthw.so, which should have a function named ZN7android15createOMXPluginEv(android::createOMXPlugin()) or createOMXPlugin (since ICS, both are tried, earlier, only the former was tried), that returns an OMXPluginBase pointer when called.

This library, libstagefrighthw.so, is vendor specific. Examples of implementations of this library, wrapping a normal OMX IL core, are available in:

  • hardware/qcom/media/libstagefrighthw/QComOMXPlugin.cpp
  • hardware/ti/omap3/libstagefrighthw/TIOMXPlugin.cpp
  • hardware/ti/omap4xxx/libstagefrighthw/TIOMXPlugin.cpp (ICS only)

All of these are very similar they simply load an OMX core from a shared library, load the OMX core function pointers from the library, and map the OMXPluginBase methods to the OMX core.

For software codecs, there’s no full proper OMX IL core in ICS, all decoders are mapped straight from the OMXPluginBase, implemented in frameworks/base/media/libstagefright/omx/SoftOMXPlugin.cpp. All software encoders (and in gingerbread, all software decoders too) are hooked up directly from the OMXCodec class, where the direct constructors of the software encoder classes are called, without any indirection via OMX like interfaces these software codecs implement the MediaSource interface directly. Due to this, these codecs cannot be accessed via the IOMX layer.

The development seems to be moving towards the OMX interfaces namely, in ICS, the software decoders have been converted to use OMX. The new ACodec class in ICS only uses codecs via the OMX interfaces.

Adding a custom OpenMAX IL plugin

Implement an OMX core for the codecs

Examples:

  • hardware/qcom/media/mm-core/omxcore
  • hardware/ti/omap3/omx/system/src/openmax_il
  • hardware/ti/omap4xxx/domx/omx_core (ICS only)
  • device/samsung/crespo/sec_mm/sec_omx/sec_omx_core (Gingerbread only)

The OMX core acts as a registry for the available codecs. If the build configuration already contains a vendor specific OMX core, the new OMX components can either be added to this OMX core, or a new separate OMX core can be added.

Build libstagefrighthw

Build libstagefright, containing an OMXPluginBase implementation that loads the OMX core. Examples:

  • hardware/qcom/media/libstagefrighthw/QComOMXPlugin.cpp
  • hardware/ti/omap3/libstagefrighthw/TIOMXPlugin.cpp
  • hardware/ti/omap4xxx/libstagefrighthw/TIOMXPlugin.cpp (ICS only)
  • device/samsung/crespo/libstagefrighthw/SEC OMX Plugin.cpp (Gingerbread only)

If one chooses to add a second OMX core, a second libstagefrighthw (with a slightly different name) has to be created, and this also requires modifications to frameworks/base/media/libstagefright/omx/OMXMaster.cpp, in order to be able to handle two different vendor libraries at the same time. Therefore, it’s probably best to integrate new components within the current vendor OMX core if one already exists.

Implement OMX components for the codecs

This is the actual wrapping of the codecs. Simple examples (which don’t use the proper real OMX API but only stagefright’s own internal API) are available in

  1. frameworks/base/media/libstagefright/omx/SoftOMXComponent.cpp

  2. frameworks/base/media/libstagefright/omx/SimpleSoftOMXComponent.cpp

  3. frameworks/base/media/libstagefright/codecs/aacdec/SoftAAC.cpp

These contain all the logic any OMX component needs to have, but the external API isn’t the proper public OMX version, but only stagefright-internal ones. Real implementations with the proper public OMX API are available in e.g.:

  • hardware/ti/omap3/omx/audio/src/openmax il/aac_dec
  • hardware/qcom/media/mm-video/vidc/vdec
  • device/samsung/crespo/sec_mm/sec_omx/sec_omx-component/video/dec/h264dec (Gingerbread only)
  • http://omxil.sourceforge.net/ a full software OMX IL implementation
Register the new OMX components

Register the new OMX components in frameworks/base/media/libstagefright/OMXCodec.cpp. Simply add the component names and the mime types it handles in the kDecoderInfo/kEncoderInfo tables. (OMXCodec in stagefright doesn’t query for which components implement certain roles, but blindly checks for all the components listed to support a certain mime type, and uses the first one that actually exists.)

   Telephony

In this section, we’ll explore the inter-workings of the telephony stack. We’ll start with the standard Android Phone app, and trace the execution of placing a call all the way down to RIL daemon.

Telephony Manager

Android Framework provides the Telephony Manager class in android.telephony package. The Telephony Manager allows you to monitor the state of the mobile network connection. However, Telephony Manager does not allow you to place or manage any calls. Only the Phone app can initiate and answer phone calls.

Phone App

Note Some of this content comes from the actual AOSP source code, licensed under Apache 2 License.
images/TelephonyArchitecture.svg
Android Telephony Architecture Diagram
Overview

Phone app does come with some UI components, such as the dialer, but the most significant part is its handling of CALL and CALL_PRIVILEGED intents to do the actual dialing of a number.

OutgoingCallBroadcaster

OutgoingCallBroadcaster activity receives CALL and CALL_PRIVILEGED Intents, and broadcasts the ACTION_NEW_OUTGOING_CALL intent which allows other applications to monitor, redirect, or prevent the outgoing call. After the other applications have had a chance to see the ACTION_NEW_OUTGOING_CALL intent, it finally reaches the OutgoingCallReceiver, which passes the (possibly modified) intent on to the SipCallOptionHandler, which will ultimately start the call using the CallController.placeCall() API.

CallController

CallController handler is the phone app module in charge of call control. This is a singleton object which acts as the interface to the telephony layer (and other parts of the Android framework) for all user-initiated telephony functionality, like making outgoing calls.

This functionality includes things like:

  • actually running the placeCall() method and handling errors or retries
  • running the whole "emergency call in airplane mode" sequence
  • running the state machine of MMI sequences
  • restoring/resetting mute and speaker state when a new call starts
  • updating the proximity sensor wake lock state
  • resolving what the voicemail: intent should mean (and making the call)

The single CallController instance stays around forever; it’s not tied to the lifecycle of any particular Activity (like the InCallScreen). There’s also no implementation of onscreen UI here (that’s all in InCallScreen).

Note that this class does not handle asynchronous events from the telephony layer, like reacting to an incoming call; see CallNotifier for that. This class purely handles actions initiated by the user, like outgoing calls.

placeCall(Intent intent) initiates an outgoing call.

Here’s the most typical outgoing call sequence:

  1. OutgoingCallBroadcaster receives a CALL intent and sends the NEW_OUTGOING_CALL broadcast.

  2. The broadcast finally reaches OutgoingCallReceiver, which stashes away a copy of the original CALL intent and launches SipCallOptionHandler.

  3. SipCallOptionHandler decides whether this is a PSTN or SIP call (and in some cases brings up a dialog to let the user choose), and ultimately calls CallController.placeCall() (from the setResultAndFinish() method) with the stashed-away intent from step (2) as the "intent" parameter.

  4. Here in CallController.placeCall() we read the phone number or SIP address out of the intent and actually initiate the call, and simultaneously launch the InCallScreen to display the in-call UI.

  5. We handle various errors by directing the InCallScreen to display error messages or dialogs (via the InCallUiState "pending call status code" flag), and in some cases we also sometimes continue working in the background to resolve the problem (like in the case of an emergency call while in airplane mode). Any time that some onscreen indication to the user needs to change, we update the "status dialog" info in the inCallUiState and (re)launch the InCallScreen to make sure it’s visible.

PhoneUtils

PhoneUtils.placeCall(Context context, Phone phone, String number, Uri contactRef, boolean isEmergencyCall, Uri gatewayUri) dials the number using the phone passed in.

CallManager

CallManager, defined in PhoneApp class.

CallManager class provides an abstract layer for PhoneApp to access and control calls. It implements Phone interface.

CallManager provides call and connection control as well as channel capability.

There are three categories of APIs CallManager provided

  1. Call control and operation, such as dial() and hangup()

  2. Channel capabilities, such as CanConference()

  3. Register notification

public Connection dial(Phone phone, String dialString) in CallManager initiate a new voice connection. This happens asynchronously, so you cannot assume the audio path is connected (or a call index has been assigned) until PhoneStateChanged notification has occurred.

dial() uses PhoneProxy to implement Phone interface to make the call.

Phone Interface

Phone interface specifies the capabilities of the underlying phone system, weather GSM, CDMA, or SIP. We’ll assume GMS from here on.

GSMPhone

GSMPhone extends PhoneBase which in turn implements the Phone interface.

It uses GSMCallTracker to dial via CommandsInterface. This interface is implemented by RIL class.

RIL in turn uses RILRequest to send the requests to rild daemon.

Tip You can use adb logcat -b radio to see the radio-specific log messages.
RIL (Java)

RIL class is the implementation of the CommandsInterface that GSMPhone uses to dial out. This Java implementation writes out the commands to RIL daemon, the native code.

RIL uses RILSender and RILReceiver to send and receive messages from rild - the RIL Daemon. Unlike most of the other Android system services, RIL uses sockets for this inter-process communication, and not the Binder.

RIL Daemon

The RIL consists of two primary components:

  • RIL Daemon: The RIL daemon initializes the Vendor RIL, processes all communication from Android telephony services, and dispatches calls to the Vendor RIL as solicited commands.
  • Vendor RIL: The radio-specific Vendor RIL of ril.h that processes all communication with radio hardware and dispatches calls to the RIL Daemon (rild) through unsolicited commands.
Note Some of the following documentation comes from once available Android Platform Development Kit: Radio Layer Interface documentation. This documentation is no longer readily available from Google (deemed outdated) but is licensed under Apache 2.0 license. We’ve updated the references as originally provided.
RIL Initialization

Android initializes the telephony stack and the Vendor RIL at startup as described in the sequence below:

  1. RIL daemon reads rild.lib path and rild.libargs system properties to determine the Vendor RIL library to use and any initialization arguments to provide to the Vendor RIL.

  2. RIL daemon loads the Vendor RIL library and calls RIL_Init to initialize the RIL and obtain a reference to RIL functions.

  3. RIL daemon calls RIL_register on the Android telephony stack, providing a reference to the Vendor RIL functions

Details of this implementation are available in AOSP/hardware/ril/rild/rild.c.

RIL Interaction

There are two forms of communication that the RIL handles:

  • Solicited commands: Solicited commands originated by RIL lib, such as DIAL and HANGUP.
  • Unsolicited responses: Unsolicited responses that originate from the baseband, such as CALL_STATE_CHANGED and NEW_SMS.
Solicited RIL Commands

The following snippet illustrates the interface for solicited commands:

  • void OnRequest (int request_id, void *data, size_t datalen, RIL_Token t);
  • void OnRequestComplete (RIL_Token t, RIL_Error e, void *response, size_t responselen);
images/RIL-solicited-commands.png
Example of solicited RIL commands

There are over sixty solicited commands grouped by the following families:

  • SIM PIN, IO, and IMSI/IMEI (11)
  • Call status and handling (dial, answer, mute…) (16)
  • Network status query (4)
  • Network setting (barring, forwarding, selection…) (12)
  • SMS (3)
  • PDP connection (4)
  • Power and reset (2)
  • Supplementary Services (5)
  • Vendor defined and support (4)
Unsolicited RIL Commands

The following snippet illustrates the interface for unsolicited commands:

  • void OnUnsolicitedResponse (int unsolResponse, void *data, size_t datalen);
images/RIL-unsolicited-commands.png
Example of unsolicited RIL commands

There are over ten unsolicited commands grouped by the following families:

  • Network status changed (4)
  • New SMS notify (3)
  • New USSD notify (2)
  • Signal strength or time changed (2)
Implementing the RIL

To implement a radio-specific RIL, create a shared library that implements a set of functions required by Android to process radio requests. The required functions are defined in the RIL header (AOSP/hardware/ril/include/telephony/ril.h).

The Android radio interface is radio-agnostic and the Vendor RIL can use any protocol to communicate with the radio. Android provides a reference Vendor RIL, using the Hayes AT command set, that you can use as a quick start for telephony testing and a guide for commercial vendor RILs. The source code for the reference RIL is found at AOSP/hardware/ril/reference-ril.

Compile your Vendor RIL as a shared library using the convention libril-<companyname>-<RIL version>.so, for example, libril-acme-124.so, where:

  • libril: all vendor RIL implementations start with libril
  • <companyname>: a company-specific abbreviation
  • <RIL version>: RIL version number
  • so: standard file extension for shared objects in Linux

For reference implementation of RIL, see AOSP/hardware/ril/libril.

Links

   Device Policy Service

images/DevicePolicyServiceArchitecture.svg
Device Policy Service Architecture

   Android Display Stack

images/SurfaceArchitecture.svg
Surface
images/SurfaceFlingerArchitecture.svg
Surface Flinger

Resources

   Camera Service

See Exposing the Android Camera Stack from 2012 Android Builders Summit by Balwinder Kaur and Joe Rickson from Aptina Imaging, Inc.

Links

   NFC Service

images/NfcServiceArchitecture.svg
NFC Service Architecture
Note This service is deployed as an application!

   Review Questions

  1. Name at least five application framework services?

  2. What is the most typical communication channel for a client to communicate with its service?

  3. What’s the difference between managers and services?

  4. How do we get the list of all application framework services?

  5. What is the purpose of the servicemanager process?

  6. How do clients/services get access to the servicemanager process?

  7. What role does JNI play in most application framework services?

  8. What is the life-cycle of most application framework services?

  9. What is the relationship between application framework services and daemons?

  10. How do most services communicate with their daemons?

  11. What is the purpose of installd and who uses it and when?

  12. What is the purpose of wpa_supplicant and who uses it?

  13. Unlike the Vibrator (or Power) service, the Alarm service does not have what?

  14. How do services (like Location service) provide call-backs to their clients?

  15. What is AudioFlinger and what is its purpose?

  16. What is the role of Audio Policy Service?

  17. What is Stagefright and what is its purpose?

  18. How do hardware manufactures take advantage of Stagefright?

  19. How does the telephony stack differ from most other services?

  20. What is SurfaceFlinger and what is its purpose?

  21. How does NFC Service differ from most other services?

Customizing Android

Note The following steps assume building Android ICS 4.0.3/4 for ARMv7 on Ubuntu 10.04 x86_64 and that the Android sources are available under ~/ics/ directory (for the current user).
Note The complete code (which we’ll develop in this section) for Marakana Alpha is available at https://github.com/marakana/alpha
The code for Marakana Alpha SDK add-on is available https://github.com/marakana/alpha_sdk_addon

   Setting up Custom Device Directory Structure

While we could overlay our device’s custom components over the existing AOSP source tree, that makes it harder to deal with future OS upgrades. Instead, we will create a self-contained directory structure to host our device.

  1. Create our vendor (e.g. mararkana) directory 

    $ mkdir device/marakana/

  2. Now create our device (e.g. alpha) sub-directory: 

    $ mkdir device/marakana/alpha

   Registering our Device with Android’s Build System

We now want to add our device to the Android’s lunch list

Note Remember that $ source build/envsetup.sh registers lunch combos that we can later build

  1. Create vendorsetup.sh file for our device (the name of this file is fixed):

    device/marakana/alpha/vendorsetup.sh
    add_lunch_combo full_marakana_alpha-eng

  2. Re-build the lunch list: 

    $ source build/envsetup.sh
including device/marakana/alpha/vendorsetup.sh
including device/moto/stingray/vendorsetup.sh
including device/moto/wingray/vendorsetup.sh
including device/samsung/crespo4g/vendorsetup.sh
including device/samsung/crespo/vendorsetup.sh
including device/samsung/maguro/vendorsetup.sh
including device/samsung/torospr/vendorsetup.sh
including device/samsung/toro/vendorsetup.sh
including device/samsung/tuna/vendorsetup.sh
including device/ti/panda/vendorsetup.sh
including sdk/bash_completion/adb.bash

  3. Finally, we can check to see that our device now appear in the lunch menu: 

    $ lunch

You're building on Linux

Lunch menu... pick a combo:
     1. full-eng
     2. full_x86-eng
     3. vbox_x86-eng
     4. full_marakana_alpha-eng
     5. full_stingray-userdebug
     6. full_wingray-userdebug
     7. full_crespo4g-userdebug
     8. full_crespo-userdebug
     9. full_maguro-userdebug
     10. full_torospr-userdebug
     11. full_toro-userdebug
     12. full_tuna-userdebug
     13. full_panda-eng

Which would you like? [full-eng]
    • We are not yet ready to select it here, because we have not yet provided the necessary makefiles for full_marakana_alpha. If we do, we’ll get: 
      $ lunch full_marakana_alpha-eng
build/core/product_config.mk:203: *** No matches for product "full_marakana_alpha".  Stop.

** Don't have a product spec for: 'full_marakana_alpha'
** Do you have the right repo manifest?

   Adding the Makefile Plumbing for our Device

We now need to add basic support for building our device.

  1. Start by creating a our own AndroidProducts.mk file, which simply defines the actual makefiles to be used when building our device:

    device/marakana/alpha/AndroidProducts.mk
    PRODUCT_MAKEFILES := $(LOCAL_DIR)/full_alpha.mk
    Note The only purpose of AndroidProducts.mk file (whose name is fixed) is to set PRODUCT_MAKEFILES to a list of product makefiles to expose to the build system. The only external variable it can use is LOCAL_DIR, whose value will be automatically set to the directory containing this file.

  2. Create the main build-file for our device (we call it full_alpha following the example from device/samsung/crespo/full_crespo.mk):

    device/marakana/alpha/full_alpha.mk
    # Defines a list of languages to be supported by our device
$(call inherit-product, $(SRC_TARGET_DIR)/product/languages_small.mk)

# Defines the rules for building the base Android platform,
# but itself is not specialized for any particular device
$(call inherit-product, $(SRC_TARGET_DIR)/product/generic.mk)

# Discard inherited values and use our own instead.
PRODUCT_NAME := full_marakana_alpha
PRODUCT_DEVICE := alpha
PRODUCT_MODEL := Full Marakana Alpha Image for Emulator
    Note
    A note inheriting from an existing product

    Our device inherits from the generic product:

    $(call inherit-product, $(SRC_TARGET_DIR)/product/generic.mk)

    which resolves to build/target/product/generic.mk file.

    This generic product (file) in turn inherits from:

    • build/target/product/generic_no_telephony.mk
      • frameworks/base/data/fonts/fonts.mk
      • external/lohit-fonts/fonts.mk
      • frameworks/base/data/keyboards/keyboards.mk
      • build/target/product/core.mk
    • build/target/product/telephony.mk

    The combination of these files set up compilation rules that define what gets included in the final product.

  3. Next, we’ll import some boiler-plate make/support files from the "generic" board - since our device will run on the emulator: 

    $ cp build/target/board/generic/BoardConfig.mk device/marakana/alpha/.
$ cp build/target/board/generic/AndroidBoard.mk device/marakana/alpha/.
$ cp build/target/board/generic/device.mk device/marakana/alpha/.
$ cp build/target/board/generic/system.prop device/marakana/alpha/.
    • AndroidBoard.mk - is essentially empty
    • device.mk - sets up basic emulator-specific configuration files
    • system.prop - used to set system-wide properties (here, just RIL settings for the emulator)
    • BoardConfig.mk - defines our device board’s kernel/hardware capabilities:
      device/marakana/alpha/BoardConfig.mk
      # config.mk
#
# Product-specific compile-time definitions.
#

# The generic product target doesn't have any hardware-specific pieces.
TARGET_NO_BOOTLOADER := true
TARGET_NO_KERNEL := true

# Note: we build the platform images for ARMv7-A _without_ NEON.
#
# Technically, the emulator supports ARMv7-A _and_ NEON instructions, but
# emulated NEON code paths typically ends up 2x slower than the normal C code
# it is supposed to replace (unlike on real devices where it is 2x to 3x
# faster).
#
# What this means is that the platform image will not use NEON code paths
# that are slower to emulate. On the other hand, it is possible to emulate
# application code generated with the NDK that uses NEON in the emulator.
#
TARGET_ARCH_VARIANT := armv7-a
TARGET_CPU_ABI := armeabi-v7a
TARGET_CPU_ABI2 := armeabi

HAVE_HTC_AUDIO_DRIVER := true
BOARD_USES_GENERIC_AUDIO := true

# no hardware camera
USE_CAMERA_STUB := true

# Set /system/bin/sh to ash, not mksh, to make sure we can switch back.
TARGET_SHELL := ash

# Enable dex-preoptimization to speed up the first boot sequence
# of an SDK AVD. Note that this operation only works on Linux for now
ifeq ($(HOST_OS),linux)
WITH_DEXPREOPT := true
endif

# Build OpenGLES emulation guest and host libraries
BUILD_EMULATOR_OPENGL := true

# Build and enable the OpenGL ES View renderer. When running on the emulator,
# the GLES renderer disables itself if host GL acceleration isn't available.
USE_OPENGL_RENDERER := true
      Note
      A note about CPU Architecture

      Because we copied build/target/board/generic/BoardConfig.mk we inherited the following:

      TARGET_ARCH_VARIANT := armv7-a
TARGET_CPU_ABI := armeabi-v7a
TARGET_CPU_ABI2 := armeabi

      Alternatively, we could have built our image for x86, either by copying build/target/board/generic_x86/BoardConfig.mk or by changing our own device/marakana/alpha/BoardConfig.mk to say the following:

      TARGET_ARCH := x86
TARGET_ARCH_VARIANT := x86-atom
TARGET_CPU_ABI := x86

   Generating Our Own Platform Signing Keys (Recommended)

Before we compile our device, we should generate our own platform signing keys [Platform_Keys], otherwise our device will not pass CTS.

  1. Define our subject/issuer info: 

    $ SIGNER="/C=US/ST=California/L=San Francisco/O=Marakana Inc./OU=Android/CN=Android Platform Signer/emailAddress=android@marakana.com"

  2. Remove the existing keys (it does not hurt to back them up first!): 

    $ rm build/target/product/security/*.p*

  3. Generate the platform key: 

    $ echo | development/tools/make_key build/target/product/security/platform "$SIGNER"
creating build/target/product/security/platform.pk8 with no password
Generating RSA private key, 2048 bit long modulus
....................+++
..........................................................+++
e is 3 (0x3)

  4. Generate the shared key: 

    $ echo | development/tools/make_key build/target/product/security/shared "$SIGNER"
creating build/target/product/security/shared.pk8 with no password
Generating RSA private key, 2048 bit long modulus
..................................................................................................+++
............+++
e is 3 (0x3)

  5. Generate the media key: 

    $ echo | development/tools/make_key build/target/product/security/media "$SIGNER"
creating build/target/product/security/media.pk8 with no password
Generating RSA private key, 2048 bit long modulus
...................+++
....................+++
e is 3 (0x3)

  6. Generate the testkey key: 

    $ echo | development/tools/make_key build/target/product/security/testkey "$SIGNER"
creating build/target/product/security/testkey.pk8 with no password
Generating RSA private key, 2048 bit long modulus
....................+++
................................................+++
e is 3 (0x3)

  7. Verify that our keys have been created: 

    $ ls -1 build/target/product/security/*.p*
build/target/product/security/media.pk8
build/target/product/security/media.x509.pem
build/target/product/security/platform.pk8
build/target/product/security/platform.x509.pem
build/target/product/security/shared.pk8
build/target/product/security/shared.x509.pem
build/target/product/security/testkey.pk8
build/target/product/security/testkey.x509.pem

  8. Check that our specific subject/issuer has been used: 

    $ openssl x509 -noout -subject -issuer -in  build/target/product/security/platform.x509.pem
subject= /C=US/ST=California/L=San Francisco/O=Marakana Inc./OU=Android/CN=Android Platform Signer/emailAddress=android@marakana.com
issuer= /C=US/ST=California/L=San Francisco/O=Marakana Inc./OU=Android/CN=Android Platform Signer/emailAddress=android@marakana.com
    Caution The build/target/product/security/.pk8 files are the private keys (.x509.pem are the certificates), and we need to make sure to keep them safe and secure - especially since we did not encrypt them!

   Building our Device For The First Time

We are now ready to try out our "custom" device - though there is nothing truly custom yet, except for the PRODUCT_* settings and our own product keys.

  1. For good measure, re-register our device: 

    $ source build/envsetup.sh
including device/marakana/alpha/vendorsetup.sh
…

  2. Now we can lunch of our device: 

    $ lunch full_marakana_alpha-eng

============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=4.0.3
TARGET_PRODUCT=full_marakana_alpha
TARGET_BUILD_VARIANT=eng
TARGET_BUILD_TYPE=release
TARGET_BUILD_APPS=
TARGET_ARCH=arm
TARGET_ARCH_VARIANT=armv7-a
HOST_ARCH=x86
HOST_OS=linux
HOST_BUILD_TYPE=release
BUILD_ID=IML74K
============================================

  3. We can now compile our device: 

    $ export USE_CCACHE=1
$ make -j10
…
Installed file list: out/target/product/alpha/installed-files.txt
Target system fs image: out/target/product/alpha/obj/PACKAGING/systemimage_intermediates/system.img
Install system fs image: out/target/product/alpha/system.img

  4. Finally, we can run it 

    $ out/host/linux-x86/bin/emulator &
    Note Replace linux with darwin for Mac OS X

  5. And we should see

    screens/MarakanaAlpha-AboutPhone-v1.png

   Adding a Custom Kernel to our Device

Our device would work fine with the provided QEMU-based (i.e. emulator-specific) kernel:

In this section, we will configure a custom kernel ([Android_Building_Linux_Kernel]) so that we can:

The steps outlined here are similar to [Android_Building_Linux_Kernel_for_Emulator]:

  1. Create a directory to host our kernel sources: 

    $ mkdir ~/kernel/
$ cd ~/kernel/

  2. Clone the Goldfish kernel sources: 

    $ git clone https://android.googlesource.com/kernel/goldfish.git
$ cd goldfish/

  3. Check out android-goldfish-2.6.29 branch of the kernel we wish to build: 

    $ git checkout -t remotes/origin/android-goldfish-2.6.29

  4. Create the default kernel configuration file (.config): 

    $ make goldfish_armv7_defconfig ARCH=arm
    Tip Depending on TARGET_ARCH_VARIANT that we specified in device/marakana/alpha/BoardConfig.mk:
    to compile for ARMv5, we would run make goldfish_defconfig ARCH=arm instead.
    to compile for x86, we would run make goldfish_defconfig ARCH=x86 instead.

  5. Change the Hardware name from "Goldfish" to "Marakana Alpha Board":

    arch/arm/mach-goldfish/board-goldfish.c:
    …
MACHINE_START(GOLDFISH, "Marakana Alpha Board")
…
    Note
    A note about the hardware (board) name

    As we already know, when the Android kernel loads, it runs init, which configures itself from init.rc and init.hardware_.rc configuration files.

    The hardware name is extracted from /proc/cpuinfo:

    $ adb shell cat /proc/cpuinfo |grep Hardware
Hardware    : Goldfish

    Basically, system/core/init/init.c:main() uses system/core/init/util.c:get_hardware_name(…) to parse /proc/cpuinfo and extract the hardware name. This name is converted to lower-case and all the space characters are trimmed.

    Android’s ueventd process (started by init) also configures itself in a similar way, by loading ueventd.rc and ueventd.hardware.rc configuration files.

    Since in our case we changed the name of our hardware to "Marakana Alpha Board", we will also need to create the corresponding: init.marakanaalphaboard.rc and ueventd.marakanaalphaboard.rc files.

  6. Configure the kernel 

    $ make menuconfig ARCH=arm
    Tip For x86, run $ make menuconfig ARCH=x86 instead.

    1. Set General setup → Local version to -marakana-alpha-release

    2. Select Enable loadable module support and within it also select Module unloading and within that Forced module unloading

    3. Customize the rest as desired

  7. Compile the kernel 

    $ make -j10 ARCH=arm CROSS_COMPILE=~/ics/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin/arm-eabi-
    Tip For x86, run make -j10 ARCH=x86 CROSS_COMPILE=~/ics/prebuilt/linux-x86/toolchain/i686-unknown-linux-gnu-4.2.1/bin/i686-unknown-linux-gnu- instead.
    Warning We ran into issues (clobbered PIC register) last time we tried compiling for x86 (January 2012) with alternative i686 toolchains.
    Note Remember that ~/ics/ is the root of the Android Open Source Project (AOSP) source tree
    Tip AOSP comes with a convenient script to compile the Linux kernel for QEMU-based emulator: ~/ics/external/qemu/distrib/build-kernel.sh, but this picked the wrong toolchain for x86.

  8. Copy the compiled kernel to our device’s alpha/ directory: 

    $ cp arch/arm/boot/zImage ~/ics/device/marakana/alpha/kernel
    Tip For x86, copy arch/x86/boot/bzImage instead.
    Note In case you were not able to compile the kernel, you could also download a pre-built one from https://github.com/marakana/alpha/raw/master/kernel

  9. Now we can go back to the AOSP source 

    $ cd ~/ics/

  10. Enable our custom kernel in BoardConfig.mk (double-negation, nice!):

    device/marakana/alpha/BoardConfig.mk
    …
TARGET_NO_KERNEL := false
…
    Note This will cause the build system to rebuild the boot.img file, but because we are still working with the emulator, we’ll have to explicitly reference our kernel on startup.

  11. Now we are ready to create our own init and ueventd configuration files by copying the ones from Goldfish: 

    $ cp system/core/rootdir/etc/init.goldfish.rc device/marakana/alpha/init.marakanaalphaboard.rc
$ cp system/core/rootdir/etc/ueventd.goldfish.rc device/marakana/alpha/ueventd.marakanaalphaboard.rc
    Note Remember, marakanaalphaboard is our device’s new hardware name (lowercased and stripped of spaces).

  12. Create a common.mk makefile for our common components, where we’ll configure our kernel and copy the init and ueventd configuration files:

    device/marakana/alpha/common.mk
    # Since this file can also be referenced by alpha-sdk_addon
# we cannot assume LOCAL_PATH points to the directory where
# this file is located. Instead, we create another variable
# to capture this directory.
MY_PATH := $(LOCAL_PATH)/../alpha

# Include all makefiles in sub-directories (one level deep)
include $(call all-subdir-makefiles)

# Enable our custom kernel
LOCAL_KERNEL := $(MY_PATH)/kernel
PRODUCT_COPY_FILES += $(LOCAL_KERNEL):kernel

# Copy our init and ueventd configuration files to the root
# file system (ramdisk.img -> boot.img)
PRODUCT_COPY_FILES += $(MY_PATH)/init.marakanaalphaboard.rc:root/init.marakanaalphaboard.rc
PRODUCT_COPY_FILES += $(MY_PATH)/ueventd.marakanaalphaboard.rc:root/ueventd.marakanaalphaboard.rc

  13. Now, we need to include our common common.mk file in our device’s main makefile (full_alpha.mk):

    device/marakana/alpha/full_alpha.mk
    …
# Include the common definitions and packages
include $(LOCAL_PATH)/common.mk

  14. Re-build our ROM, which, in this case just creates a new out/target/product/alpha/boot.img (that we won’t actually use): 

    $ make -j10
…

  15. Restart the emulator (with our new kernel) 

    $ out/host/linux-x86/bin/emulator -kernel out/target/product/alpha/kernel &
    Note We have to run our emulator with -kernel, because otherwise it would ignore the kernel in our newly built boot.img and instead use the default one (prebuilt/android-arm/kernel/kernel-qemu-armv7). This applies just to the emulator.

  16. Test 

    $ adb shell cat /proc/version
Linux version 2.6.29-marakana-alpha-release-g46b05b2 (sasa@thermal) (gcc version 4.4.3 (GCC) ) #3 Tue Jan 3 13:25:44 PST 2012
    Note The path to adb was added to our PATH when we $ source build/envsetup.sh (on a Linux host, it comes from out/host/linux-x86/bin/adb)

  17. We could also take a look at the updated About screen:

    screens/MarakanaAlpha-AboutPhone-v2.png

   Adding a Custom Native Library and Executable to our Device

   Using our Native Library via a Custom Daemon

   Exposing our Native Library via Java (i.e. JNI)

   Consuming our a Custom Java/JNI→Native Library via a Custom App (Optional)

   Exposing our Custom Library via a Custom IPC/Binder Service

   Building a Custom App Using a Custom Service Manager

   Creating a Custom SDK Add-on (Optional)

Distributing our Custom SDK Add-on (Optional)

  1. First, we need to create a repository.xml file to describe our add-on and publish it to our server:

    https://marakana.com/external/android/sdk-addon/repository.xml
    <?xml version="1.0" encoding="UTF-8"?>

<sdk:sdk-addon
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:sdk="http://schemas.android.com/sdk/android/addon/1">

    <sdk:add-on>
        <sdk:name>Alpha Add-On</sdk:name>
        <sdk:api-level>15</sdk:api-level>
        <sdk:vendor>Marakana</sdk:vendor>
        <sdk:revision>1</sdk:revision>
        <sdk:description>Android + Marakana Alpha Add-on, API 15, revision 1 </sdk:description>
        <sdk:desc-url>http://marakana.com/external/android/sdk-addon/</sdk:desc-url>
        <sdk:uses-license ref="marakana-android-addon-license" />
        <sdk:archives>
            <sdk:archive os="any">
                <sdk:size>96382546</sdk:size>
                <sdk:checksum type="sha1">5cec8cc3f3064441cb96a50e2b8aa528681ffc78</sdk:checksum>
                <sdk:url>marakana_alpha_sdk_addon_api-15_r1.zip</sdk:url>
            </sdk:archive>
        </sdk:archives>
        <sdk:libs>
        </sdk:libs>
    </sdk:add-on>

    <sdk:license type="text" id="marakana-android-addon-license">
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this SDK addon except in compliance with the License.
You may obtain a copy of the License at: http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
    </sdk:license>
</sdk:sdk-addon>
    Note We have to make sure that we adjust the info as necessary to reflect our particular site/addon.

  2. Next, we need to upload our addon to our site, relative to <sdk:sdk-addon> → <sdk:add-on> → <sdk:desc-url> URL (e.g. https://marakana.com/external/android/sdk-addon/marakana_alpha_sdk_addon_api-15_r1.zip)

  3. We are now ready to test it out in our Android SDK Manager

    1. Go to Tools → Manage Add-on Sites… in the menu bar

    2. Click on New…

    3. Enter the base URL in the URL field (e.g. https://marakana.com/external/android/sdk-addon/) and click on OK

    4. Click on Close to dismiss the Add-on Sites window

    5. Under Packages we should now see Alpha Add-On by Marakana, with a status Not installed

    6. Click on the checkbox next to our package

    7. Click on Install 1 package… button

    8. Accept the license terms and click on Install

  4. Test that it works by creating an AVD based on our add-on and/or using it in Eclipse

   Review Questions

  1. What is a home directory for a new custom board within AOSP source-tree?

  2. What is the purpose of vendorsetup.sh?

  3. What does $(call inherit-product, $(SRC_TARGET_DIR)/product/generic.mk) do?

  4. What is the significance of PRODUCT_PACKAGES?

  5. What is the purpose of BoardConfig.mk?

  6. What is the purpose of development/tools/make_key and why do we want to use it?

  7. What is the significance of /proc/cpuinfo?

  8. When/why do we use include $(call all-subdir-makefiles)?

  9. What is the purpose of LOCAL_MODULE?

  10. What is the purpose of LOCAL_MODULE_TAGS?

  11. How do we compile individual modules efficiently?

  12. Name at least three include $(BUILD_XXX) targets.

  13. What do we need to do in order to launch a custom daemon on startup?

  14. How do we expose custom Java system libraries to applications?

  15. How do applications consume custom Java system libraries?

  16. When do we use include $(BUILD_PREBUILT)?

  17. When/where do we use $(JNI_H_INCLUDE)?

  18. What is the purpose of {@hide}?

  19. When do we use include $(BUILD_DROIDDOC)?

  20. What do we need to do in order to register a custom service to servicemanager?

  21. When/why would we use LOCAL_JAVA_LIBRARIES += framework?

  22. When/why would we use LOCAL_CERTIFICATE := platform?

  23. What does include $(BUILD_PACKAGE) build?

  24. What is SDK add-on and when/why would we want to build/use one?

  25. How do we distribute custom SDK add-ons?

Android USB Support (Appendix)

   Overview of Android USB Support

Android USB Host and Accessory Modes

Android supports a variety of USB peripherals and Android USB accessories (hardware that implements the Android accessory protocol) through two modes: USB host and USB accessory.

In USB host mode, the Android-powered device acts as the host.

In USB accessory mode, the external USB hardware act as the USB hosts.

Android USB Support

USB accessory and host modes are directly supported in Android 3.1 (API level 12) or newer platforms.

Note Support for USB host and accessory modes are ultimately dependent on the device’s hardware, regardless of platform level. You can filter for devices that support USB host and accessory through a <uses-feature> element, as discussed later.

   Android USB Host Mode

API Overview

The android.hardware.usb package contains the following classes supporting USB host mode:

UsbManager

Allows you to enumerate and communicate with connected USB devices.

UsbDevice

Represents a connected USB device and contains methods to access its identifying information, interfaces, and endpoints.

UsbInterface

Represents an interface of a USB device, which defines a set of functionality for the device. A device can have one or more interfaces on which to communicate on.

UsbEndpoint

Represents an interface endpoint, which is a communication channel for this interface. An interface can have one or more endpoints, and usually has input and output endpoints for two-way communication with the device.

UsbDeviceConnection

Represents a connection to the device, which transfers data on endpoints. This class allows you to send data back and forth sychronously or asynchronously.

UsbRequest

Represents an asynchronous request to communicate with a device through a UsbDeviceConnection.

UsbConstants

Defines USB constants that correspond to definitions in linux/usb/ch9.h of the Linux kernel.

In general, you:

  1. Obtain a UsbManager to retrieve the desired UsbDevice

  2. Find the appropriate UsbInterface and the UsbEndpoint of that interface to communicate on

  3. Open a UsbDeviceConnection to communicate with the USB device

In most situations, you need to use all of these classes (UsbRequest is only required if you are doing asynchronous communication) when communicating with a USB device.

Using the Manifest to Require USB Support

Not all Android-powered devices are guaranteed to support the USB host APIs.

To indicate that your application requires USB host support, add the following elements to your application’s manifest:

Working with Devices

When users connect USB devices to an Android-powered device, the Android system can determine whether your application is interested in the connected device. If so, you can set up communication with the device if desired.

To do this, your application has to:

  1. Discover connected USB devices either by:

    • Using an intent filter to be notified when the user connects a USB device, or
    • Enumerating USB devices that are already connected.

  2. Ask the user for permission to connect to the USB device, if not already obtained.

  3. Communicate with the USB device by reading and writing data on the appropriate interface endpoints.

Discovering a Device Using an Intent Filter, The Manifest

To have your application discover a particular USB device, you can specify an intent filter to filter for the android.hardware.usb.action.USB_DEVICE_ATTACHED intent.

In your activity, you can obtain the UsbDevice that represents the attached device from the intent like this:

UsbDevice device = (UsbDevice) intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);

The following example shows how to declare the intent filter in your application’s manifest:

<activity ...>
    ...
    <intent-filter>
        <action android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED" />
    </intent-filter>

    <meta-data android:name="android.hardware.usb.action.USB_DEVICE_ATTACHED"
        android:resource="@xml/device_filter" />
</activity>

The <meta-data> element points to an external XML resource file that declares identifying information about the device that you want to detect.

Discovering a Device Using an Intent Filter, The Resource File

In the XML resource file, declare <usb-device> elements for the USB devices that you want to filter.

Save the resource file in your application’s res/xml/ directory.

Enumerating USB Devices

If your application is interested in inspecting all of the USB devices currently connected while your application is running, it can enumerate devices on the bus.

UsbManager manager = (UsbManager) getSystemService(Context.USB_SERVICE);
// ...
HashMap<String, UsbDevice> deviceList = manager.getDeviceList();
UsbDevice device = deviceList.get("deviceName");

If desired, you can also just obtain an iterator from the hash map and process each device one by one:

UsbManager manager = (UsbManager) getSystemService(Context.USB_SERVICE);
// ...
HashMap<String, UsbDevice> deviceList = manager.getDeviceList();
Iterator<UsbDevice> deviceIterator = deviceList.values().iterator();
while (deviceIterator.hasNext()) {
    UsbDevice device = deviceIterator.next()
    // ...
}

Obtaining Permission to Communicate With a Device

Before communicating with the USB device, your application must have permission from your users.

Note If your application uses an intent filter to discover USB devices as they’re connected, it automatically receives permission if the user allows your application to handle the intent. If not, you must request permission explicitly in your application before connecting to the device.

Explicitly asking for permission might be necessary in some situations such as when your application enumerates USB devices that are already connected and then wants to communicate with one.

To explicitly obtain permission:

  1. Call UsbManager.requestPermission().

  2. The call to requestPermission() displays a dialog to the user asking for permission to connect to the device.

  3. The system generates a broadcast intent with a boolean EXTRA_PERMISSION_GRANTED extra indicating the user’s response.

The following sample code shows how to create a broadcast receiver to process the response:

private static final String ACTION_USB_PERMISSION = "com.android.example.USB_PERMISSION";

private final BroadcastReceiver mUsbReceiver = new BroadcastReceiver() {

    public void onReceive(Context context, Intent intent) {
        String action = intent.getAction();
        if (ACTION_USB_PERMISSION.equals(action)) {
            synchronized (this) {
                UsbDevice device = (UsbDevice)intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);

                if (intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)) {
                    if (device != null) {
                      // Call method to set up device communication
                   }
                }
                else {
                    Log.d(TAG, "permission denied for device " + device);
                }
            }
        }
    }
};

To register the broadcast receiver, add this in your onCreate() method in your activity:

IntentFilter filter = new IntentFilter(ACTION_USB_PERMISSION);
registerReceiver(mUsbReceiver, filter);

To display the dialog that asks users for permission to connect to the device, call the UsbManager.requestPermission() method:

UsbManager mUsbManager = (UsbManager) getSystemService(Context.USB_SERVICE);
UsbDevice device;
// ...
mPermissionIntent = PendingIntent.getBroadcast(this, 0, new Intent(ACTION_USB_PERMISSION), 0);
mUsbManager.requestPermission(device, mPermissionIntent);

Communicating With a Device

Communication with a USB device can be either synchronous or asynchronous.

To properly set up communication with a device, you need to obtain the appropriate UsbInterface and UsbEndpoint of the device that you want to communicate on and send requests on this endpoint with a UsbDeviceConnection. In general, your code should:

The following code snippet is a trivial way to do a synchronous data transfer. Your code should have more logic to correctly find the correct interface and endpoints to communicate on and also should do any transferring of data in a different thread than the main UI thread:

private byte[] bytes = "hello usb".getBytes();
private static int TIMEOUT = 0;
private boolean forceClaim = true;

...

UsbInterface intf = device.getInterface(0);
UsbEndpoint endpoint = intf.getEndpoint(0);
UsbDeviceConnection connection = mUsbManager.openDevice(device);
connection.claimInterface(intf, forceClaim);
connection.bulkTransfer(endpoint, bytes, bytes.length, TIMEOUT); // Do in another thread

To send data asynchronously, use the UsbRequest class to initialize and queue an asynchronous request, then wait for the result with requestWait().

Terminating Communication With a Device

When you are done communicating with a device or if the device was detached, close the UsbInterface and UsbDeviceConnection by calling releaseInterface() and close().

To listen for detached events, create a broadcast receiver like below:

BroadcastReceiver mUsbReceiver = new BroadcastReceiver() {
    public void onReceive(Context context, Intent intent) {
        String action = intent.getAction();

      if (UsbManager.ACTION_USB_DEVICE_DETACHED.equals(action)) {
            UsbDevice device = (UsbDevice)intent.getParcelableExtra(UsbManager.EXTRA_DEVICE);
            if (device != null) {
                // call your method that cleans up and closes communication with the device
            }
        }
    }
};

Creating the broadcast receiver within the application, and not the manifest, allows your application to handle detached events only while it is running.

   Android USB Accessory Mode

Choosing the Right USB Accessory APIs

The USB accessory APIs were introduced to the platform in Android 3.1. They are also available in Android 2.3.4 using the Google APIs add-on library.

API Overview

The following classes support the USB accessory APIs:

UsbManager

Allows you to enumerate and communicate with connected USB accessories.

  • If you are using the add-on library, you must obtain the UsbManager object in the following manner: 
    UsbManager manager = UsbManager.getInstance(this);
  • If you are not using the add-on library, you must obtain the UsbManager object in the following manner: 
    UsbManager manager = (UsbManager) getSystemService(Context.USB_SERVICE);
UsbAccessory

Represents a USB accessory and contains methods to access its identifying information.

  • When you filter for a connected accessory with an intent filter, the UsbAccessory object is contained inside the intent that is passed to your application. If you are using the add-on library, you must obtain the UsbAccessory object in the following manner: 
    UsbAccessory accessory = UsbManager.getAccessory(intent);
  • If you are not using the add-on library, you must obtain the UsbAccessory object in the following manner: 
    UsbAccessory accessory = (UsbAccessory) intent.getParcelableExtra(UsbManager.EXTRA_ACCESSORY);

Because the add-on library is a wrapper for the framework APIs, the classes that support the USB accessory feature are similar. You can use the reference documentation for the android.hardware.usb even if you are using the add-on library.

Android Manifest Requirements

Not all Android-powered devices are guaranteed to support the USB accessory APIs.

To indicate that your application requires USB accessory support, add the following elements to your application’s manifest:

Working with Accessories

When users connect USB accessories to an Android-powered device, the Android system can determine whether your application is interested in the connected accessory. If so, you can set up communication with the accessory if desired.

To do this, your application has to:

  1. Discover connected accessories by using an intent filter that filters for accessory attached events or by enumerating connected accessories and finding the appropriate one.

  2. Ask the user for permission to communicate with the accessory, if not already obtained.

  3. Communicate with the accessory by reading and writing data on the appropriate interface endpoints.

Discovering a Device Using an Intent Filter, The Manifest

To have your application discover a particular USB device, you can specify an intent filter to filter for the android.hardware.usb.action.USB_ACCESSORY_ATTACHED intent.

In your activity, you can obtain the UsbAccessory that represents the attached device from the intent.

The following example shows how to declare the intent filter in your application’s manifest:

<activity ...>
    ...
    <intent-filter>
        <action android:name="android.hardware.usb.action.USB_ACCESSORY_ATTACHED" />
    </intent-filter>

    <meta-data android:name="android.hardware.usb.action.USB_ACCESSORY_ATTACHED"
        android:resource="@xml/accessory_filter" />
</activity>

The <meta-data> element points to an external XML resource file that declares identifying information about the device that you want to detect.

Discovering a Device Using an Intent Filter, The Resource File

In the XML resource file, declare <usb-accessory> elements for the accessories that you want to filter.

Save the resource file in your application’s res/xml/ directory.

Enumerating Accessories

You can have your application enumerate accessories that have identified themselves while your application is running.

Use the UsbManager.getAccessoryList() method to get an array all the USB accessories that are connected:

UsbManager manager = (UsbManager) getSystemService(Context.USB_SERVICE);
UsbAccessory[] accessoryList = manager.getAcccessoryList();
Note Currently, only one connected accessory is supported at one time, but the API is designed to support multiple accessories in the future.

Obtaining Permission to Communicate With an Accessory

Before communicating with the USB accessory, your application must have permission from your users.

Note If your application uses an intent filter to discover USB devices as they’re connected, it automatically receives permission if the user allows your application to handle the intent. If not, you must request permission explicitly in your application before connecting to the device.

Explicitly asking for permission might be necessary in some situations such as when your application enumerates USB devices that are already connected and then wants to communicate with one.

To explicitly obtain permission:

  1. Call UsbManager.requestPermission().

  2. The call to requestPermission() displays a dialog to the user asking for permission to connect to the accessory.

  3. The system generates a broadcast intent with a boolean EXTRA_PERMISSION_GRANTED extra indicating the user’s response.

The following sample code shows how to create a broadcast receiver to process the response:

private static final String ACTION_USB_PERMISSION = "com.android.example.USB_PERMISSION";

private final BroadcastReceiver mUsbReceiver = new BroadcastReceiver() {

    public void onReceive(Context context, Intent intent) {
        String action = intent.getAction();
        if (ACTION_USB_PERMISSION.equals(action)) {
            synchronized (this) {
            UsbAccessory accessory = (UsbAccessory) intent.getParcelableExtra(UsbManager.EXTRA_ACCESSORY);

                if (intent.getBooleanExtra(UsbManager.EXTRA_PERMISSION_GRANTED, false)) {
                    if (accessory != null) {
                      // Call method to set up accessory communication
                   }
                }
                else {
                    Log.d(TAG, "permission denied for accessory " + accessory);
                }
            }
        }
    }
};

To register the broadcast receiver, add this in your onCreate() method in your activity:

IntentFilter filter = new IntentFilter(ACTION_USB_PERMISSION);
registerReceiver(mUsbReceiver, filter);

To display the dialog that asks users for permission to connect to the device, call the UsbManager.requestPermission() method:

UsbManager mUsbManager = (UsbManager) getSystemService(Context.USB_SERVICE);
UsbAccessory accessory;
// ...
mPermissionIntent = PendingIntent.getBroadcast(this, 0, new Intent(ACTION_USB_PERMISSION), 0);
mUsbManager.requestPermission(accessory, mPermissionIntent);

Communicating With an Accessory

You can communicate with the accessory by using the UsbManager to obtain a file descriptor that you can set up input and output streams to read and write data to descriptor.

UsbAccessory mAccessory;
ParcelFileDescriptor mFileDescriptor;
FileInputStream mInputStream;
FileOutputStream mOutputStream;

// ...

private void openAccessory() {
    Log.d(TAG, "openAccessory: " + accessory);
    mFileDescriptor = mUsbManager.openAccessory(mAccessory);
    if (mFileDescriptor != null) {
        FileDescriptor fd = mFileDescriptor.getFileDescriptor();
        mInputStream = new FileInputStream(fd);
        mOutputStream = new FileOutputStream(fd);
        Thread thread = new Thread(null, this, "AccessoryThread");
        thread.start();
    }
}

In the thread’s run() method, you can read and write to the accessory by using the FileInputStream or FileOutputStream objects.

When reading data from an accessory with a FileInputStream object, ensure that the buffer that you use is big enough to store the USB packet data.

Note At a lower level, the packets are 64 bytes for USB full-speed accessories and 512 bytes for USB high-speed accessories. The Android accessory protocol bundles the packets together for both speeds into one logical packet for simplicity.

Terminating Communication With an Accessory

When you are done communicating with an accessory or if the accessory was detached, close the file descriptor that you opened by calling close().

To listen for detached events, create a broadcast receiver like below:

BroadcastReceiver mUsbReceiver = new BroadcastReceiver() {
    public void onReceive(Context context, Intent intent) {
        String action = intent.getAction();

        if (UsbManager.ACTION_USB_ACCESSORY_DETACHED.equals(action)) {
            UsbAccessory accessory = (UsbAccessory)intent.getParcelableExtra(UsbManager.EXTRA_ACCESSORY);
            if (accessory != null) {
                // call your method that cleans up and closes communication with the accessory
            }
        }
    }
};

Creating the broadcast receiver within the application, and not the manifest, allows your application to handle detached events only while it is running.

The Android Open Accessory Development Kit

The Android Open Accessory Development Kit (ADK) provides an implementation of an Android USB accessory that is based on the Arduino open source electronics prototyping platform.

The main hardware and software components of the ADK include:

Legal Disclaimer

INFORMATION IN THIS DOCUMENT IS PROVIDED “AS IS”. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. MARAKANA ASSUMES NO LIABILITY WHATSOEVER AND MARAKANA DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO THIS INFORMATION INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

Other names and brands may be claimed as the property of others.

Copyright © 2012 Marakana Inc.

← →

/

#